JBoss 4 Direction Shown with Roadmap Document

Discussions

News: JBoss 4 Direction Shown with Roadmap Document

  1. JBoss 4 Direction Shown with Roadmap Document (197 messages)

    Bill Burke and Scott Stark have authored a document that details the direction for JBoss 4. This road map document contains items for the short term (J2EE 1.4 compliance), and the long term (POJO AOP). Here you will see what it is going to take JBoss to get to compliance, and what they want to see as "Aspect-Oriented Middleware".

    Download the JBoss 4.0 Road Map (pdf)

    Threaded Messages (197)

  2. Impressive[ Go to top ]

    Wow. To think that all of this highly complex work is being done by people who may never have seen each other and talked to each other or without anticipation of monetary gain is truly mind boggling. Go!
  3. Impressive?[ Go to top ]

    What, vaporware is impressive?

    It's useful that they actually _have_ a plan - that alone is a very nice change of pace. I imagine several high-profile customers of theirs insisted on seeing a real plan and not an e-mail of bullet points. But beyond the nice formatting, what does it tell us?

    Overall progress is listed as 20% for the "short term plan", with a Q2 2004 target. Completing the remaining 80% in six months? Sure, this is possible if they sacrifice features or quality. I don't see them doing 80% of the work in 6 months and getting a really high quality implementation that meets all of their aims.

    They probably realize this - the individual items are all marked "open" for "Release Target Date". I'd say in reality they'll either release something crappy in June, or something reasonable sometime in 2005. Hopefully the fact that all of the individual items were listed in the "spec" phase was just an oversight - there is hopefully some development going on right now, not just design and spec reading :-)

    FYI, the overall progress chart:

       7 items at 0%
       1 item at 5%
       1 item at 10%
       7 items at 20%
       1 item at 30%
       2 items at 40%
       1 item at 60%

    A devilish application of bad statistics shows the average completion of the effort at 16.25%.

    This covers the "short term" effort. Add in the fact that JBoss Group LLC people are expected to be billing 50% of their time, and when do you expect to see any of this completed?

    On a personal note, I find it interesting that a complete re-write of their JMS, with a focus on "Highly Available JMS. JMS with fault tolerance. No need for load balancing and other clustered scalability enhancements in the short term. Just reliability" is estimated as "Preliminary development estimate for JBoss/JMS is 1 man month". 1 person working for 22 business days is going to write a JMS provider from scratch with an emphasis on reliability? Damn! I guess if scalability and performance is completely thrown out the window it is possible - but who wants the worlds most reliable JMS implementation that chugs along at 5 messages/second?

    Overall, it's very brave of the JBoss people to publish such a plan publically, but I don't know how realistic it is. Their short term plan looks like several man-years of effort to me, so I'm not sure why anyone would even bother reading the long-term plan at this point in time (coming soon in 2017 :-).

         -Mike
  4. Impressive?[ Go to top ]

    I agree it is very ambitious. But they have abundant manpower. There are coders out there working on it even if though they are not JBoss group employees.

    So, they do have "several man-years" or at least can begin to approach it. The real challenge is coordinating all that effort and yet delivering a quality product. Going by the quality of commercial products out there, I wouldn't blame JBoss for quality.
  5. Impressive?[ Go to top ]

    \code freedom\
    I agree it is very ambitious. But they have abundant manpower. There are coders out there working on it even if though they are not JBoss group employees.
    \code freedom\

    For ambitious new coding, I doubt much of the "abundant manpower" is applicable. I'd say 90% of the design and coding effort will be handled by 10 people or less, and almost all of them will be JBoss Group employees.

    You can confirm this by looking at CVS histories and seeing who's checking in major feature changes. Out of the 84 (?) commiters, the majority of them aren't going to contribute much to JBoss 4.0 until it's feature-complete. At that point maybe you'll see a spike in activity - fixing bugs can often be done by the hoarde, implementing major features from scratch generally comes down to 1 or 2 people per major subsystem. I'd say the majority of the "abundant manpower" will be maintaining the 3.x branch, not working on pie-in-the-sky 4.0 stuff.

    Plus, loss of the people who have defected to Geronimo only exacerbates this problem.

    Going through the CVS histories also show other interesting trends. Typically, nothing at all happens in a subsystem for many months. Then one or two people get into a frenzy of activity for 3-6 weeks, and immediately after something major pops out. Then it goes quiescient again. You can see this in the cache areas, you can see it in the AOP areas, you see it in the JMS areas - frantic activity, big lulls, frantic activity, big lulls. This matches a model of a small number of primary movers and shakers who make big changes, then get pulled off to work on other things, then come back to their baby many months later, spin for a bit reacquainting themselves with the code base, then bashing out another frantic bit of stuff. The first casualty of such a model is consistency, followed closely by robustness. Oh, the features may get done, but in a very staccato fashion. This is why I don't think you're going to see a rich and robust 4.0 in Q2 2004 - too few people in charge of too much code who are on again/off again in their main responsibilities. You don't work part time over 6 months, or full time for a month, on something like a robust JMS implementation and expect stellar results. Now take that JMS example and multiply it by 15 or so. Throw in people screaming (sometimes literally) "screw 4.0, I want that feature in 3.2 now!!" and things slow down even more.

    This is why most commercial offerings have dedicated development teams who don't go out on billing - to keep them focused. There are some benefits to having such people out in the field to gain real-world experience, but doing it 6 months out of the year every year for all of you primary product developers hurts ya more than it helps, IMHO.

         -Mike
  6. Open Source vs Commercial Vendor[ Go to top ]

    jBoss inovated Hibreante, AOP, MicroKernal. What did Commercail Vendors do other than stay in a EJB heard?
    Plus you get the benefit of OSS (No budget meeting, no legal, higher quality, no access to source should vendor change bus. plans, etc.) Why wait to get a PO?
    Unless your CTO is golfing with the sales people, why not help you organization be more effective

    Put in Tomcat and jOnas, that is a lot of choice, and competition is good even Commercail Vendors should compete. I use Tomcat just fine, and according to Netcraft, Tomcat is the # one deployed in market share.

    As far as man power, several very experienced developers can out code a large group of juionr developers, just out of school developers. They also have a community and are not ashamed to show thier code so somone in the community can point it out. Plus they use their own stuff, as opposed to torw it over the wall.

    You can develop a .war on any of the containers, and the deploy on some other container that you fnid more effective, that is the point of Java, cross platform. We all test when deploying right?

    Without choice and a user community you can use .net, Java has a diferent culture.

    .V
  7. Open Source vs Commercial Vendor[ Go to top ]

    \Vic\
    jBoss inovated Hibreante, AOP, MicroKernal. What did Commercail Vendors do other than stay in a EJB heard?
    \Vic\

    What ya been smoking lately, Vic? JBoss effectively bought Hibernate (by hiring Gavin King), they've copied AOP from AspectJ et al, and MicroKernel architectures have been all the rage for quite a few decades now. JBoss' primary claim to fame is for having created a free J2EE container e.g. writing code against someone else's spec. There has been some interesting innovations in implementation choices along the way, but from what I've seen they were all created people who have since moved on away from JBoss.

    \Vic\
    [Rambling JBoss-Jboss-he's-the-man-if-he-can't-beat-em-no-one-can]
    \Vic\

    I think you lost the thread a bit here. The only thing my post was trying to convey was that they've outlined a huge amount of work and apparently allocated only 6 months in which to do it all - in addition to other obligations. Talk all you want about the benefits of open source, but it doesn't work miracles. Whether code is open, closed, or wearing a peek-a-boo bra doesn't change the fact that software of a certain complexity takes _time_ to develop. Whether you're IBM, BEA, or Joe Open Source, or even a gaggling horde of Joe Open Sourcers, you don't bang out a JMS provider in 3 or 4 weeks.

    As for choice - yeah, it's nice that some people are doing open source projects. What does that have to do with JBoss' development plan? Does the fact that JBoss is open source or gives "choice" somehow magically bend the realities of creating complex software? If I give my code away for free do 8 month coding efforts magically shrink to 5 weeks?

    Fact is, JBoss is behind the curve now and has lost time puttering about with questionable AOP stuff and marketing activities. IBM has an early release 1.4 impl available now, and 1.4 is Geronimo's target. With these pressures, JBoss will probably keep people out on 50% billing anyway. This isn't a recipe for getting a JBoss 1.4 J2EE implementation by summer 2004.

    Open source isn't a golden key that opens the kingdom of heaven. Open source may help expand choice - but that doesn't mean that the open source alternative will actually be the best choice. No, just like elsewhere in real life, some open source will be good, some great, and most of it will suck - just like commercial closed software.

        -Mike
  8. Open Source vs Commercial Vendor[ Go to top ]

    "Whether you're IBM, BEA, or Joe Open Source, or even a gaggling horde of Joe Open Sourcers, you don't bang out a JMS provider in 3 or 4 weeks."

    This seems like an open challenge to the JBoss jedi knights. Let's see if they are upto the mark. I look forward to your analysis of their JMS implementation when they are done.
  9. Open Source vs Commercial Vendor[ Go to top ]

    \code freedom\
    This seems like an open challenge to the JBoss jedi knights. Let's see if they are upto the mark. I look forward to your analysis of their JMS implementation when they are done.
    \code freedom\

    OK, they've got a man-month. Start your clocks, and I'll see you all on Feb. 1st (see, I'm even throwing in a couple of extra days for free!). :-)

        -Mike
  10. CVS Trolling...[ Go to top ]

    Some trolling through the JBoss CVS highlights the staccato development nature. Note that this isn't a diss to any of the authors - it's just the facts, ma'am. This is against the CVS head...

    For JMS
    (http://cvs.sourceforge.net/viewcvs.py/jboss/jboss-jms/src/main/org/jboss/jms/)

    Most files last checked in 4 months ago, a few 3 months ago. Many files have the checkin comment "Getting there, something actually works now ".

    For AOP Framework:
    (http://cvs.sourceforge.net/viewcvs.py/jboss/jboss-aop/src/main/org/jboss/aop/)

    Most files were last touched by Bill Burke 5 months ago. Minor touchups, like fixing imports, were done quite recently - 4 months ago.

    For enterprise "aspects":
    (http://cvs.sourceforge.net/viewcvs.py/jboss/jboss-aspects/src/main/org/jboss/aop/remoting/)

    Most recent changes 5 months ago.

    For cache:
    (http://cvs.sourceforge.net/viewcvs.py/jboss/jboss-cache/src/main/org/jboss/cache/)

    Much more active - there was a big push in November to get stuff out for a demo of the caching stuff. Most files were last touched 3-4 weeks ago, a few were touched a couple of days ago. Sporadic updates before that, big multi-month gaps prior to September.

    -----------------------------------------------

    I think the above pretty well outlines a trend :-) Troll back further through the logs, and in other areas, and you see big pushes and then big lulls in development. You also see other trends - like fewer commits from "big names", and more commits from lesser known individuals, which may very well coincide with JBoss Group shucking them out to make dollars (indeed, most of cache appears to be written/commited by bwang00, not belaban - I guess Bela's going more of an architect role on this).

    There's nothing "wrong" with any of this - it's their work, their lives, they're free to do what they like. But the CVS trends seem to strongly contradict any evidence that JBoss 4.0 development will complete in Q2 2004.

         -Mike
  11. CVS Trolling...[ Go to top ]

    You also see other trends - like fewer commits from "big names", and more

    > commits from lesser known individuals, which may very well coincide with JBoss
    > Group shucking them out to make dollars (indeed, most of cache appears to be
    > written/commited by bwang00, not belaban - I guess Bela's going more of an
    > architect role on this).

    The reason for this is that I'm working on multiple projects at the same time: currently I'm working on a new transport for JGroups, which dynamically discovers all interfaces and multicast on all of them. This is configurable though. The advantage is that we will get rid of a common problem, and where nodes in a cluster don't find each other in multi homed machines . This is a source of confusion for many users, and providing this kind of Transport will take some load off of my shoulders.

    In parallel I'm working on designing new features into the TreeCache, and I'm working with Ben ( who has taken over TreeCacheAop). You can check out a preliminary to do list at http://cvs.sourceforge.net/viewcvs.py/jboss/jboss-cache/docs/design/todo.txt?rev=1.4&view=auto.

    this is in no particular order yet, but it will give you the big picture of what we will add to the TreeCache.

    After finishing the new JGroups transport, I will work on session replication for Tomcat 5 , based on the TreeCacheAop. this will give us both a useful piece of functionality, namely session replication, and requirements for TreeCacheAop. TreeCacheAop is not done yet, and session replication is a great driver of requirements for it.

    We recently got access to HP's labs, and will stress test TreeCache and TreeCacheAop with regard to performance.

    So, development on TreeCache and TreeCacheAop hasn't come to a standstill, but we are actively working on it.
    Hope this helps,


    Bela
  12. CVS Trolling...[ Go to top ]

    The reason for this is that I'm working on multiple projects at the same time: currently I'm working on a new transport for JGroups, which dynamically discovers all interfaces and multicast on all of them. This is configurable though. The advantage is that we will get rid of a common problem, and where nodes in a cluster don't find each other in multi homed machines . This is a source of confusion for many users, and providing this kind of Transport will take some load off of my shoulders.


    I wonder if those users might not be desktop users. A regular sys admin would set up the network interfaces so he knows what's going on. As a potential user, I would not expect my app server to decide what interfaces to use or not use.

    Just my 2c.
  13. CVS Trolling...[ Go to top ]

    I wonder if those users might not be desktop users. A regular sys admin would set up the network interfaces so he knows what's going on.


    Not all JBoss users are sysadmins/netadmins... Actually I found that a lot of people even don't know what a multi-homed system is.

    > As a potential user, I would not expect my app server to decide what interfaces to use or not use.

    The motivation here is to provide something that works, without having to determine bind interfaces first.

    Bela
  14. CVS Trolling...[ Go to top ]

    Not all JBoss users are sysadmins/netadmins... Actually I found that a lot of people even don't know what a multi-homed system is.


    Maybe I am not a typical developer, but to me if someone undertands transactional distributed caches then they should also understand what a network interface is, including virtual ones.

    Also, in a regular LAN there is a switch, which usually needs to be configured to route multicast packets. Even more importantly, the SA should be told ahead of time if you are going to make heavy use of the network. But I guess this still doesn't apply to users who run a single instance (but then do they need multicasting?).

    > > As a potential user, I would not expect my app server to decide what interfaces to use or not use.
    >
    > The motivation here is to provide something that works, without having to determine bind interfaces first.

    I guess it's a difference in philosophy.
  15. CVS Trolling...[ Go to top ]

    Maybe I am not a typical developer, but to me if someone undertands transactional distributed caches then they should also understand what a network interface is, including virtual ones.

    >
    > Also, in a regular LAN there is a switch, which usually needs to be configured to route multicast packets. Even more importantly, the SA should be told ahead of time if you are going to make heavy use of the network. But I guess this still doesn't apply to users who run a single instance (but then do they need multicasting?).
    >
    > > > As a potential user, I would not expect my app server to decide what interfaces to use or not use.
    > >
    > > The motivation here is to provide something that works, without having to determine bind interfaces first.
    >
    > I guess it's a difference in philosophy.

    yes and no. Philosophy means taste usually and taste usually isn't right or wrong it is taste. In configuration you have right and wrong (I won't even qualify it with imho :). It is called use cases thinking.

    Analyse configuration with "how do people use that stuff". I love statistical use cases. Imagine you have 1000 users. You want to maximize the happy function on that distribution (I *am* a math major). Where is the maximum usage? 90% is all the developers and people who put this thing together, or people who test bla bla bla on LANs. For them you want the dumb down approach, the one that doesn't break, the fool proof one, not the one where you need to touch 3 files, understand 10 pages of material and get approval from your SA bla bla bla, all that to get JBossCache to boot. They take it out of the box and it works, traffic will show a multicast for 10 ms in discovery (BIG FUCKING DEAL) and we are done with discovery. End of story, already some real life people say on this thread "It would have saved me HOURS of pulling my hair out". Excellent.

    Then there is 5% that does prod on LAN, that have a real life deployment. They supposedly went through development/testing/prod and YES if they understand distributed caches they should know about virtual network interfaces and the cost of the default configuration. But the point IS THAT WE GET IN THE FACE OF THIS PRODUCTION MINORITY, NOT THE BLISSFULLY IGNORANT MAJORITY. That is the use case of our stuff. Ease of use FIRST.

    marcf
  16. CVS Trolling...[ Go to top ]

    \MF\
    Analyse configuration with "how do people use that stuff". I love statistical use cases.
    \MF\

    Of course you do Marc, they give you such a fertile ground to ignore reality and just make stuff up as you go along.

    \MF\
    Imagine you have 1000 users. You want to maximize the happy function on that distribution (I *am* a math major). Where is the maximum usage? 90% is all the developers and people who put this thing together, or people who test bla bla bla on LANs. For them you want the dumb down approach, the one that doesn't break, the fool proof one, not the one where you need to touch 3 files, understand 10 pages of material and get approval from your SA bla bla bla, all that to get JBossCache to boot. They take it out of the box and it works, traffic will show a multicast for 10 ms in discovery (BIG FUCKING DEAL) and we are done with discovery.
    \MF\

    What a load of bullshit! Let's run some more realistic statistics. 90% of your users do not use clustering. Period. Out of the remaining 10% (and I'm being _very_ generous here), I'll be generous and say 10% of those people trying out clustering for the hell of it (but not in production) have multi-homed system.

    So you're talking about 1% of your user base that may be affected.

    Then let's talk about the horrible, awful pain you're inflicting here. No, it's not 3 files and 10 pages of material.

    It's. One. IP. Address.

    1% of your users have to specify an IP address in one configuration file. Oh, the horror! Oh, the shame!!

    \MF\
    They take it out of the box and it works, traffic will show a multicast for 10 ms in discovery (BIG FUCKING DEAL) and we are done with discovery.
    \MF\

    Sorry, wrong, won't work. Bela's magic code will continue multicasting on all interfaces it finds (other than loopback). It has to in order to deal with order of servers coming up - there might not be another server to listen to, yet, and so he has to keep the multicast port open everywhere.

    You pay a performance hit for this, hitting each multicast port serially, and will also possibly piss off people for multicasting on networks you have no business multicasting on.

    \MF\
     End of story, already some real life people say on this thread "It would have saved me HOURS of pulling my hair out". Excellent.
    \MF\

    Marc, one guy said his system configuration was wrong. One guy, with the name of "code freedom", who just happens to use the same dumb phrases you're so fond of. Why don't you just come out and say "Yeah, I'm the stupid guy who was pulling my hair out" :-)

    \MF\
    Then there is 5% that does prod on LAN, that have a real life deployment. They supposedly went through development/testing/prod and YES if they understand distributed caches they should know about virtual network interfaces and the cost of the default configuration. But the point IS THAT WE GET IN THE FACE OF THIS PRODUCTION MINORITY, NOT THE BLISSFULLY IGNORANT MAJORITY. That is the use case of our stuff. Ease of use FIRST.
    \MF\

    No, with this nice little protocol you won't "get in the face of" anyone anywhere. This problem will be caught, but often this sort of thing just slips through the cracks. Quite often, it takes awhile for production people to wrap their heads around a new system, and they'll accept what the developers give them initially with the assumption that "well, it seems to work". It's not until the thing cooks for awhile in production that someone might notice some odd behavior. And at that point, the issue is _much_ harder to diagnose and often involves tons of people to figure out.

        -Mike
  17. to Mike Spille[ Go to top ]

    Thanks for taking your time to write such interesting posts... We all get tired of marketing stuff here :)

    happy new year to all!!
  18. Dumb phrases[ Go to top ]

    "One guy, with the name of "code freedom", who just happens to use the same dumb phrases you're so fond of"

    What dumb phrase did I use? Can you please point out?
  19. CVS Trolling...[ Go to top ]

    Ease of use FIRST.


    That's what I meant by 'a difference in philosophy'. I guess you stated your own taste/philosophy above: "ease of use first".

    I am just not that kind of guy because I *like* learning how things work. And I also know from experience that the guys who don't know how things work don't make good developers.
  20. Multi Homed Systems[ Go to top ]

    \Bela Ban\
    The reason for this is that I'm working on multiple projects at the same time: currently I'm working on a new transport for JGroups, which dynamically discovers all interfaces and multicast on all of them. This is configurable though. The advantage is that we will get rid of a common problem, and where nodes in a cluster don't find each other in multi homed machines . This is a source of confusion for many users, and providing this kind of Transport will take some load off of my shoulders.
    \Bela Ban\

    How bizarre. In my experience, the times you would want to do this border closely on "never" - and the very few times you do you want to think it through very, very carefully.

    Any machine that is multi-homed, either via multiple NICs or via some virtual mechanism, is set up that way for a very good reason. On a home NAT setup, you might say "the Internet's that-a-way on W.X.Y.Z, internal network is this-a-way on A.B.C.D". In a corporate situation, it might be "eth0 goes out to the regular LAN, eth1 is a gigabit connection to the mainframe", or something similar.

    In any case, most of the time if you're directing any kind of networking traffic, you really want to know where you're directing it to. For a NAT situation, you _really_ want to know if you're trying to multicast to your local servers or if you're trying to push out to the Internet. For a corp situation, you really really want to know if you're trying to get to the mainframe or the regular LAN (or however your setup happens to work).

    Mebbe it's just me, but everyone who has a multi-homed system has run into this issue countless times. And they know that whenever they do any networking thing, they have to point out the right interface, via some GUI, or control panel, or a configuration file.

    Finding all interfaces and multicasting on all of them? Sounds like both a waste of development time, and a tool that's more dangerous than useful to me.

        -Mike
  21. Direct customer interaction[ Go to top ]


    > Mebbe it's just me, but everyone who has a multi-homed system has run into this issue countless times. And they know that whenever they do any networking thing, they have to point out the right interface, via some GUI, or control panel, or a configuration file.
    >


    Yes it's just you. Bela wouldn't be doing this if users actually understood what you so clearly and brilliantly do. It is one of the most common problems that Bela troubleshoots, right Bela?

    This is an example why I love our development process. Being out in the field either through consulting or answering support requests is a great way to learn user requirements and see how people use our software. Sure, it slows down our release schedule sometimes, but our product is free and we don't need to depend on revenue from upgrade licenses like our competitors do.

    Bill
  22. Direct customer interaction[ Go to top ]

    \Bill Burke\
    Yes it's just you. Bela wouldn't be doing this if users actually understood what you so clearly and brilliantly do. It is one of the most common problems that Bela troubleshoots, right Bela?
    \Bill Burke\

    So tell me, what do these same hapless users do when they set up a web server or regular app server instance, and have to specify a interface for the listening port to go on? Do you just say "screw it" and open up HTTP on every interface? Er, no, you don't. The person setting up the server has to know which interface they want to listen on.

    \Bill Burke\
    This is an example why I love our development process. Being out in the field either through consulting or answering support requests is a great way to learn user requirements and see how people use our software. Sure, it slows down our release schedule sometimes, but our product is free and we don't need to depend on revenue from upgrade licenses like our competitors do.
    \Bill Burke\

    You love working for customers who can't determine which IP address to use on a multi-homed system? You _encourage_ such people to run multicasting cluster software on their networks?

    "Please Mr. Jboss Man, specifying an IP address in a configuration file is _too hard_. Our lead developer just says these numbers are all gobbledygook to him. And he's rather upset - he insists these aren't proper numbers anyway, look - there's 3 decimal points!!! Whoever heard of a number with three decimal points? We're not sure what he's smoking in that cellar cubicle of his, but he may have a point. Please, please just spray packets on any hapless network connection you find! Here, here I have memos from our networking comms people and our security group begging us to do this!! Whether the machine has 1 NICK or 2 NICK's (who is this Nick guy anyway?!?!?), or even no NICKs, we want you multicasting from here to Grandma's house!

    In fact, we're tired of specifying HTTP listen ports too. Our SVP _insists_ that JBoss just listen on every available port on the system - this is much better than typing the number '80' in an XML file.

    The enterprise architect would also like to eliminate all file and directory information from your software - please change the code to search all drives and directories for configuration information, and to write all log output to every disk drive you can find connected. Our developers are just too dumb, and frankly can't be trusted to find the right disk.

    Oh, and John in sales would really prefer that JBoss just installs itself on every machine like a virus - just eliminate the whole installation and setup process altogether - an auto-propogating JBoss will save us hundreds of thousands of dollars in admin costs.

    Hmmm...oh, yes, I almost forgot. Our CEO wants you to re-write JBoss in assembler. He took a course on VAX assembler in '81 and he still has his notebook showing how much faster assembly code is than these so-called high-level languages. So please re-write your server in assembly. Oh, do it in microcode if you can, but our CEO is an understanding guy and he'll take assembler if that's the best you can do.

    Lastly, for 6.0 we want JBoss on a 1"x1" chip that we can just plug into the network, and then go out to the Four Seasons and celebrate.

    As our thanks for this effort, the enterprise architect has spent the $30 for your documentation, and will send his admin assistants nephew to one of your training camps (he's a whiz on his gamecube, after your training we're going to fire the lead developer and move Johnny into his place!). We hope this will more than cover your development expenditure."

        -Mike
  23. Mike Spille[ Go to top ]

    Mike,
    I take it you are anti jBoss in any way, and that you do not see any pluses.

    What do you think is a more effective or a better alternative that you would propose people evaluate, and what are the points that you like on it?

    Just want to learn.
    .V
  24. Mike Spille[ Go to top ]

    \Vic Cekvenich\
    I take it you are anti jBoss in any way, and that you do not see any pluses.
    \Vic Cekvenich\

    That's not the case at all - JBoss is very useful as a free J2EE container. Alot of people get alot of mileage out of it. But they also run into trouble when they find the claims of certain JBoss individuals, the Web site, and some documentation run contrary to what the code is actually capable of.

    The JBoss codebase in and of itself is useful - the fork to Elba proves that if nothing else does. But claims about future directions can't be evaluated on the basis of code, because the code doesn't exist yet. So we have to look at past development efforts, look at the plans in detail, and see how closely all that matches with reality. In this case, look at the JBoss 4.0 roadmap, the detailed functionality therein, look at the people who will be doing that work, and look at past CVS activity as a broad gauge of how future development might progress.

    We see a very aggressive roadmap - a 6 month timeline and a ton of complex features to be fit in it. We see most areas targetted by that timeline have been sitting fallow for several months in CVS. Key figures think that taking time off from must-have features of this timeline for something like multicasting on all network interfaces on a machine is a useful activity (as opposed to forcing us poor developers to specify an IP address in a file). For all the touting of the value of experience out in the field, the reality is that this field work and extra-curricular activities is resulting in little actual coding work.

    You may quite validly say "Who cares?". Many people probably don't. But there are others who are planning projects of their own, and they may be looking to use things specified in this roadmap. The burning question for them is, do I take the risk and think these things will come out in a high-quality implementation on the timelines scheduled - and that I'll be able to use them in my own work - or should I disbelieve that the features I need will be there when I need 'em? In less dense phraseology - should I wait for JBoss to do it, or should I start shopping for an alternative now?

    \Vic Cekvenich\
    What do you think is a more effective or a better alternative that you would propose people evaluate, and what are the points that you like on it?
    \Vic Cekvenich\

    Well, first people have to know what their own goals are. What are they looking for in an application server? What J2EE components are they going to use? How far are they gonna push the spec? How much do they value scalability, reliability, and hitting very recent specifications? What extra-spec items do they need?

    Now, what can deliver these goods for me? It may be that paying a licensing hit and going Websphere may be a better bet for people targetting J2EE 1.4. You may like WebLogic's built-in JMS provider, recognize that JBoss' leaves a lot to be desired, and again take a licensing hit. Maybe you find that for your own needs that JoNas is "as good" as JBoss, and has alot less controversy surrounding it. Maybe you're a risk taker way out on the edge, and you look around and see alot more actual work being done on Geronimo than JBoss, and decide trusting the reputation of the developers therein and their ability to deliver is a better bet than risking it all on Bill Burke and Marc Fleury.

    Maybe, just maybe, if you have enough resources to back you, you fork JBoss yourself and bet your own people on developing the features you need.

    The best thing I can tell you is to look at CVS and make your own decisions. Look at the things that JBoss keeps screaming to the rafters about - AOP, J2EE-less enterprise software, mo' recently J2EE 1.4 - and then look and see how the CVS tree reflects those priorities. Do their words match their code? In my research - they don't. Most of the stuff that JBoss people are screaming their heads off about have been sitting in CVS, un-updated, for months.

    You see, it's not that I'm anti-JBoss. It would be more accurate to say I'm anti- false propoganda. You might say that I trust the reality of the codebase over press releases and developers' promises and Marc Fleury's cussing. I trust my own testing over others' words.

    Alot of people have an enormous ability to make very credible - even exciting - pronouncements about what "can" be done. But these words don't mean a damn thing when you see "return new Xid[0]" as your recovery mechanism, an NPE thrown out of AOP code for simple scenarios, catch blocks that print a stack trace and nothing more, and the newest file in your subsystem of greatest interest last being updated in August.

         -Mike
  25. Direct customer interaction[ Go to top ]

    Funny but you are beginning to see everything from JBoss negatively.

    We have had problems due to multi-homed systems at least once, though it wasn't anything to do with an appserver. Our Informix server would refuse to connect to it's replicated partner, although everything was setup correctly. The problem turned out to be that the sysadmin had setup both IP addresses on the machine to have the same hostname. Of course, it was the sysadmin's fault but he did not look at the /etc/hosts file. He was busy checking out static routes, calling the network folks to check their routes on the router, etc. We could have saved an hour in downtime.

    The point is that if this helps at least one idiot customer save hours of precious downtime, it might be worth it. Your point is well taken only if this negatively impacts product quality for those who do not have issues due to multi-homed systems.
  26. Direct customer interaction[ Go to top ]

    I think I see, "code freedom" (and a nod to Bill Burke, the coiner of the quoted psuedonym as an indictment). "Although everything was setup correctly", ah, um, er, not everything was set up correctly. :-)

        -Mike
  27. Direct customer interaction[ Go to top ]

    Mike

    I assure you I am not Bill Burke. I admire your intelligence and reading your posts, you would definitely make a first-rate project lead. Well, the fact that I admire your intelligence should convince you I am not Bill Burke :-)
  28. Direct customer interaction[ Go to top ]

    Never thought you were Bill Burke - he doesn't typically use the Jedi thing.

         -Mike
  29. Direct customer interaction[ Go to top ]

    Not everything was setup correctly. But that is the ground reality of large application deployments. Is this something to get religious over? It's just a different philosophical outlook. One chooses to work around such issues, the other says if things were configured correctly, this should not have happened.
  30. competition?[ Go to top ]

    Bill: .. our product is free and we don't need to depend on revenue from upgrade licenses like our competitors do.

    If you consider your competition to be other app servers, then you are competing only for bragging rights, since "winning" doesn't have any other direct benefits.

    If you are competing for developers to help with the project, then you are competing against Jonas and Geronimo and Tomcat and Jetty (etc.) since those are the other open source J2EE implementations (or parts thereof).

    Last I checked, your real business (JBoss Group LLC) built its revenue on service dollars, and since you are doing training, consulting, etc. for JBoss, that means you compete against companies that sell JBoss training and consulting, right? So your competitors don't sell "upgrade licenses" either.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Clustered JCache for Grid Computing!
  31. competition?[ Go to top ]

    Bill: .. our product is free and we don't need to depend on revenue from upgrade licenses like our competitors do.

    >
    > If you consider your competition to be other app servers, then you are competing only for bragging rights, since "winning" doesn't have any other direct benefits.
    >

    Open Source in general lives on bragging rights. JBoss inc and professional open source (RedHat, mySQL) lives on dollars. Winning in the dollar metric is the simplest metric there is.
     
    > If you are competing for developers to help with the project, then you are competing against Jonas and Geronimo and Tomcat and Jetty (etc.) since those are the other open source J2EE implementations (or parts thereof).
    >

    Cameron, you are still a bit naive as to how open source really works and you have never done it, so I am always a bit irritated when you pontificate about open source on your blog and such. Open Source, professional open source works like your company does :) In fact, face it, it is no different than the way you operate except our development is wide open, our debugging is distributed and massivelly parallel, our recruitment pool is the best in the business, our AO innovation pace is sustainable.

    Tomcat is part of JBoss so I am not competing with Tomcat :) JBoss inc offers the first and only Tomcat support in the industry AS PART OF OUR STANDARD CONTRACTS. And the lead developper is with us so we can look at a customer in the eye and say "we will support you, for real". Today JBoss inc sponsors Tomcat.

    Jetty is marginal, they voted themselves out of JBoss, Geronimo is a still born with a legal cloud before a product, nice work. Jonas is our competition today but is the french government, I am a product of that society and let me tell you the arrogance of the government let's me sleep at night.
      
    > Last I checked, your real business (JBoss Group LLC) built its revenue on service dollars, and since you are doing training, consulting, etc. for JBoss, that means you compete against companies that sell JBoss training and consulting, right? So your competitors don't sell "upgrade licenses" either.
    >

    Yes and no.

    No (simple): our product competition is our product competition. Our product competitors sells, in fact today survives in Wall street through upgrade licenses, we don't. Period. It was always a big argument for us in sales. PERIOD. Pretending otherwise is wasting TSS bandwidth and an insult to everyone's intelligence here.

    YES: JBoss inc's business is a hybrid.
    Consulting: no. We do high-end consulting, we don't do implementation (please optimize cache, help with architecture review etc) so the "consulting" part needs to be refined if you really want to talk competition.

    Training: yes and no. yes for J2EE partwe do compete with every mom and pop shop. For J2EE training (the run of the mill J2EE training) it is indifferenciated so we gave up on it, margins suck and every clown on the face of the earth is a "J2EE expert". On the advanced JBoss training, it is more clear cut, JBoss inc is the right place to teach you the inner working of JBoss. JBoss is becoming big, no one at JBoss knows ALL of it (maybe only ADRIAN is there today :) so I am always amused when two bit clowns claim to be able to teach JBoss internals. I call them crocodiles "big mouth, little hands", they have never contributed to JBoss or even to open source in general, but their mouths are big, they can talk the talk like they are experts.

    SUPPORT: NO. Finally, and that is the real business of JBoss inc., the maintenance and support of JBoss. Take your business model, it is based on license revenue and maintenance revenue, all product related. Imagine you keep the license out and keep the maintenance part. Now you got the JBoss model. JBoss Cache is free, you can buy support if you want. Will you buy support from your local SI? he doesn't know JBoss, never participated. Only if he is a JASP (JBoss Authorized Service Provider) will he have access to JBoss inc in third line. Otherwise it is snake oil.

    So you see our core business is still a product play, not a service play, in fact we kept the service component to a product play. It works, it works well enough to fuel a 30 person company and its growth. We are not talking billions and billions of revenue, but we are talking about a real company.

    marcf
  32. happy new year[ Go to top ]

    oh well,

    I realize I am probably the only LOSER posting on TSS on a DEC 31 @ 7PM.

    Anyway. Happy new year to everyone. I believe 2004 will be a big year for free software in general (RH, mySQL, JBoss).

    It is fun so far, keep it that way,

    Peace love and good code,

    marcf
  33. competition?[ Go to top ]

    Hmmm, marc, no comment on my posts? More the pity, it gave me a nice kick comin into the new year....

    \MF\
    Open Source in general lives on bragging rights. JBoss inc and professional open source (RedHat, mySQL) lives on dollars. Winning in the dollar metric is the simplest metric there is.
    \MF\

    Nice, empty meaningless phrasing there. Way to go, you're on a roll, keep it up!!

    \MF\
     Open Source, professional open source works like your company does :) In fact, face it, it is no different than the way you operate except our development is wide open, our debugging is distributed and massivelly parallel, our recruitment pool is the best in the business, our AO innovation pace is sustainable.
    \MF\

    Your recruitment results are no better than anyone else's. You've got Bela and Gavin, but you've also got Bill and Andy. And Bela and Gavin still spend some time on non-JBoss bits. Net sum loss.

    You also forget the small difference of dollars. All development on the JBoss code base, and Hibernate, and JGroups, nets you zero dollars. You are relying on secondary effects of that effort - training, support, consulting. A much dicier bizness in many ways, and far less "scalable" in terms of revenue vs. expenditure. Only a small fraction of users will buy support, and training and consulting requires highly skilled man power. The business models are worlds apart.

    \MF\
    Jetty is marginal, they voted themselves out of JBoss, Geronimo is a still born with a legal cloud before a product, nice work. Jonas is our competition today but is the french government, I am a product of that society and let me tell you the arrogance of the government let's me sleep at night.
    \MF\

    Marc, you're slipping here. Too wordy. You need a catchier sound bite. What you really, really want to say is "It's simple: if it ain't JBoss it sucks!!! Now come over here and SMD, bitch!". That's honest and direct and some people will buy it. Start whining about Jetty and Geronimo and Jonas and, well, you sound like a whiner. The mere act of naming them will drive a number of people to them - "Hmm, if Marc Fleury really bitches about it and denigrates it, it has to be good!".

    \MF\
    No (simple): our product competition is our product competition. Our product competitors sells, in fact today survives in Wall street through upgrade licenses, we don't. Period. It was always a big argument for us in sales. PERIOD. Pretending otherwise is wasting TSS bandwidth and an insult to everyone's intelligence here.
    \MF\

    Sales? What sales? You don't sell your product. You don't even _own_ a product! I'd expect you of all people to know this, but all JBoss Group sells is support contracts, training, and consulting.

    You. Do. Not. Sell. A. Product.

    Is that clear enough?

    \MF\
     JBoss is becoming big, no one at JBoss knows ALL of it (maybe only ADRIAN is there today :) so I am always amused when two bit clowns claim to be able to teach JBoss internals. I call them crocodiles "big mouth, little hands", they have never contributed to JBoss or even to open source in general, but their mouths are big, they can talk the talk like they are experts.
    \MF\

    What a load of dung. The majority of people who wrote the JBoss code do not work for JBoss Group LLC. In fact, your "expert" trainers know about as much about the code as I do. The reality is that there are more experts in CDN on JBoss internals then those who work for JBoss Group.

    And, to be frank, people will learn far more about Jboss internals from reading the code then getting "trained" by some talking head who's going to spew JBoss propaganda. The code is there for everyone to see. Why would my company pay for a talking head that's going to spout propaganda, when they can review the code for themselves?

    Oh, you'll get some suckers, but not that many. Given that you have fewer than 10 employees who are true experts in this stuff, what kind of bang can you give for the buck anyway?

    You may not like it, but there are far more experts on JBoss internals outside of your little company by an order of magnitude than are in it.

    And, of course ironically, every day your core group of "experts" spends training some poor group of schlubs, the code base that training is based on whithers more on the vine. While the JBoss Group experts yap away for dollars,
    your competitors are writing mountains of new code. Bop on over to the Geronimo developer mailing list, and see what they've done in the past month. Compare it to the anemic changes to the JBoss CVS tree in the same period of time. In the next two years you're going to have a bunch of JBoss experts when no one is using JBoss anymore.

    \MF\
    Only if he is a JASP (JBoss Authorized Service Provider) will he have access to JBoss inc in third line. Otherwise it is snake oil.
    \MF\

    As with training, some suckers will buy it. Most companies will find it a waste of money. Given your company's size, support isn't going to be exactly stellar. It's far more productive to look in the source and fix problems yourself then hope that Bill Burke will come riding in on a white horse to save the day. Your own forums show this - questions lay unanswered there for months. Free support for Websphere and Weblogic are absolutely stellar. Yours sucks DD. Given the publically visible level of free support (just this side of nil), who's going to bet on your paid support?

        -Mike
  34. competition?[ Go to top ]

    MF:
    > Jetty is marginal, they voted themselves out of JBoss, Geronimo is a
    > still born with a legal cloud before a product, nice work.

    Jetty is not marginal and we did not vote ourselves out of JBoss.
    We were kicked out for supporting other OS projects.

    However, we continue to support JBoss, which we think is a great
    application server. We now build JBoss-Jetty sars with every release of
    Jetty and they are available from the Jetty web site. In fact, it appears
    that JBoss releases are now using the sars that we prepare for your
    Jetty releases. Which is great, even if we remain deleted from the
    contributors page of JBoss!

    We also continue to support Jetty integration with JOnAS, Geronimo, Avalon
    and many other applications and platforms. The concept is called "sharing"!

    Also please stop spreading FUD about Geronimo. There is no legal cloud, as
    any concerns raised have been completely addressed and rejected. If you have
    any new concerns, please bring them to the attention of the Geronimo project.

    -gregw
  35. Multi Homed Systems[ Go to top ]

    Mike: Finding all interfaces and multicasting on all of them? Sounds like both a waste of development time, and a tool that's more dangerous than useful to me.

    If it makes it easier in "development mode" then it isn't a bad idea. In fact, it would be neat if it could also auto-detect the optimal TTL setting for that matter ;-). For "production mode," I agree that these "auto" options are just not kosher.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Clustered JCache for Grid Computing!
  36. Multi Homed Systems[ Go to top ]

    \Cameron Purdy\
    If it makes it easier in "development mode" then it isn't a bad idea.
    \Cameron Purdy\

    How many development machines are multi-homed? For those that are, how many developers don't know which interface is which?

    How's this for a fun scenario: your SO is using your main dev PC for some top-secret manipulation in the Sims. You're bored, you decide to do some work on your home's dual-NIC'd NAT gateway out to your DSL/cable modem/what have you.

    You've used the JGroups/JBoss/whatever default setup, multicast gets set up on all interfaces.

    Now you've got a nice multicast set up to your home LAN. And another identical setup to the Internet. Depending on your TTL setting, you've just put your JBoss cluster (or whatever) on the Internet. Let the fun commence!!!

    Imagine same scenario where the "other interface" (is that like the "other woman"?) hooks into a serious connection to a scarce resource on your net, possibly secure in some manner. Are the comms people really going to look favorably upon you now that you've just created a direct bridge between the corporate LAN and this special network?

    Look at other software - does Apache just auto-listen on port 80 (or 8080 or what have you) on every interface it can find? How 'bout your RDBMS? No? Think for a moment on why they don't do this....

    \Cameron Purdy\
    In fact, it would be neat if it could also auto-detect the optimal TTL setting for that matter ;-)
    \Cameron Purdy\

    In the spirit of multi-casting on all interfaces, I'd propose that TTL should be helpfully and automatically set to "128" at all times - if you're really after ease of use and making sure you always connect, you can't beat the "multicast across continents" setting!!! Anything less and it _might_ fail, and we all know how bad failures are!

        -Mike
  37. Multi Homed Systems[ Go to top ]

    I would share Mike’s humor. The idea of detecting all network interfaces and spamming all of them by default with a specific protocol is, in my opinion, just ridiculous. Spending anyone’s time on this exercise is clear testament for project "organization" and technical "leadership abilities". There’s old saying "if you cater your product to idiots – only idiots will use it"...

    Regards,
    Nikita.
    Fitech Laboratories.
  38. Multi Homed Systems[ Go to top ]

    How many development machines are multi-homed?


    Look at your laptop: you have
    - loopback
    - eth0 (or equivalent): ethernet card interface
    - eth1 (or equivalent): wifi interface

    Here's a real-life use case:
    - Linux box with 1 interface (eth0)
    - Windows laptop with 2 interfaces: eth0 (ethernet, disabled), and eth1 (wifi, enabled)

    Without setting the bind address, I will pick eth0 on Linux and eth0 on the Windows box (InetAddress.getLocalHost()). (This is different on Win2K with media sense enabled)

    Thus, the members won't find each other, unless you set the bind address on the Windows box to eth1.

    You suggest I'm wasting my time. This is not true. 80% of my support questions regarding JGroups/JBossCluster are regarding members not finding each other.
    So obviously not everyone is an expert in configuring multi homed systems...

    My goal is not to replace the default high-perf transport. I'm adding a second *user-friendly* transport.

    The goal is to have JGroups/JBossCluster work out of the box. Then - when people familiarize themselves with the technology - we will point out that they might want to switch to the high-perf transport. BTW: the new transport is as efficient as the old one if you only have 1 NIC (besides the loopback).

    So, yes, this is not work on the cool new JBoss 4 feature set, but it is important nevertheless. And it has the nice side effect of reducing my time to respond to "my nodes don't find each other" questions.
    Cheers,
    Bela
  39. Multi Homed Systems[ Go to top ]

    \Bela Ban\
    Look at your laptop: you have
    - loopback
    - eth0 (or equivalent): ethernet card interface
    - eth1 (or equivalent): wifi interface

    Here's a real-life use case:
    - Linux box with 1 interface (eth0)
    - Windows laptop with 2 interfaces: eth0 (ethernet, disabled), and eth1 (wifi, enabled)

    Without setting the bind address, I will pick eth0 on Linux and eth0 on the Windows box (InetAddress.getLocalHost()). (This is different on Win2K with media sense enabled)

    Thus, the members won't find each other, unless you set the bind address on the Windows box to eth1.
    \Bela Ban\

    How many multi-homed systems do you see? What percentage?

    Out of that small percentage - how many of them involve people who are smart enough to write code to use JGroups, but who are too dumb tell IP addresses apart?

    Keep in my my other relevant post on this - what do other products do which have to open up a listener port in non-multicast land? Why, horror of horrors, they require you to specify an IP address. Shocking!

    \Bela Ban\
    You suggest I'm wasting my time. This is not true. 80% of my support questions regarding JGroups/JBossCluster are regarding members not finding each other.
    So obviously not everyone is an expert in configuring multi homed systems...
    \Bela Ban\

    Bela, did you choose your words carefully above. You went from talking about multihomed systems to saying "80% of my support questions ....are regarding members not finding each other". You trying to tell us that 80% of your questions are "I'm too dumb to know what multi-homed means"? That it takes a tremendous amount of time to say "see the faq"?

    \Bela Ban\
    The goal is to have JGroups/JBossCluster work out of the box. Then - when people familiarize themselves with the technology - we will point out that they might want to switch to the high-perf transport. BTW: the new transport is as efficient as the old one if you only have 1 NIC (besides the loopback).

    So, yes, this is not work on the cool new JBoss 4 feature set, but it is important nevertheless. And it has the nice side effect of reducing my time to respond to "my nodes don't find each other" questions.
    \Bela Ban\

    I'm sorry Bela, but this is sheer stupidity on your part as well as that of your users. You are wasting your time. Your "user friendly transport" will get used in production by the idiots, and it will cause far more problems than having a poor ignorant soul figure out what IP address to use.

    I'll go back to one of my original arugments - why don't Oracle and Apache and everyone else take your approach? Why not provide a way for your Http Server and RDBMS and every other process to "listen everywhere"? Mostly, it's because requiring people to know the right IP address is, ahem, rather trivial. If someone is so bad with multihomed systems, how well do you think they'll do setting up Oracle, or Apache, or JBoss? I'd say their installation is going to be a disaster anyway. And better to have an immediate failure then to spray every interface. An immediate failure, people have a capacity to figure out what's wrong and learn something. Your approach? You're begging for far more problems down the road, like security breaches and networks wigging out at 2 am.

    I'll say it bluntly: this really, really is a waste of your time, and incredibly stupid to boot. This is going to haunt you for some time, trust me. People who know servers and know TCP/IP are going to frankly not go near JGroups with a 10 foot pole if they hear that the lead developer has created a stack that sprays every interface indiscriminately. I've personally respected your work in the past, but this....this is so...so amateurish. It _will_ tarnish your reputation, and people are going to start wondering what else you've cooked up along these lines inside of JGroups and JBoss. Even if there's nothing they're going to wonder. This is just incredibly lame, man. Why not just hard code a TTL of 128, and set an unchangable ping time of 2 milliseconds while you're at it?!?!?

        -Mike
  40. Multi Homed Systems[ Go to top ]

    Keep in my my other relevant post on this - what do other products do which have to open up a listener port in non-multicast land? Why, horror of horrors, they require you to specify an IP address.


    I don't care about other products. I have identified a big problem, and I'm fixing it. It is as simple as that.


    > Bela, did you choose your words carefully above. You went from talking about multihomed systems to saying "80% of my support questions ....are regarding members not finding each other". You trying to tell us that 80% of your questions are "I'm too dumb to know what multi-homed means"?


    80% of the JGroups or JBossCluster specific questions, yes.


     
    > I'm sorry Bela, but this is sheer stupidity on your part as well as that of your users. You are wasting your time. Your "user friendly transport" will get used in production by the idiots, and it will cause far more problems than having a poor ignorant soul figure out what IP address to use.


    You have the mindset of a 'Linux' hacker. Of course these people won't need this transport. But I'm talking about a huge number of people who are not at that level. Reminds me of the good old days when nobody was interested in working on the user-friendliness of Linux.


    > And better to have an immediate failure then to spray every interface.

    I don't spray every interface by default: you can set up a list of interfaces, or set it to "all", plus exclude interfaces. E.g. use "all" interfaces minus loopback (127.0.0.1).


    People who know servers and know TCP/IP are going to frankly not go near JGroups with a 10 foot pole if they hear that the lead developer has created a stack that sprays every interface indiscriminately.


    What are you so concerned about *my* reputation ? Again, this transport is a *deployment* issue, you *don't* need to use it.

    Cheers,
    Bela
  41. Multi Homed Systems[ Go to top ]

    \Bela Ban\
    I don't care about other products. I have identified a big problem, and I'm fixing it. It is as simple as that.
    \Bela Ban\

    Bela, what you've "identified" is that certain software domains require a bit of thought to work in. Some software domains actually require some intelligent input from the installers and developers. Ignoring what other products do in this instance is ignoring man-centuries of experience: no one else does this because it's been found to be an incredibly bad idea to open any kind of network port unless you need it.

    \Bela Ban\
    80% of the JGroups or JBossCluster specific questions, yes.
    \Bela Ban\

    OK, let's go right to the quick of it. Pull up the JGroups user mailing list:

       http://sourceforge.net/mailarchive/forum.php?forum_id=1724

    I looked at all of the most recent 100 messages going back from now to October. You posted 45 times, and out of those 45 times....1 post may have had to do with a multi-homed system. Your other 44 posts had nothing to do with multi-homed systems.

    It's quite clear that people using JGroups directly do not have a problem setting up JGroups on multihomed systems. 1 question in slightly over 2 months.

    Perhaps the real problem is with people using JBoss clustering (which uses JGroups). Which begs the question - JBoss people can presumably set up the listener port(s) properly for non-cluster options, why pray tell can't they do it for the multicast address?

    \Bela Ban\
    You have the mindset of a 'Linux' hacker. Of course these people won't need this transport. But I'm talking about a huge number of people who are not at that level. Reminds me of the good old days when nobody was interested in working on the user-friendliness of Linux.
    \Bela Ban\

    Being able to distinguish which IP address to use is hardly at the Linux hacker level. And anyone running any sort of server on a multihomed system consistently runs into this problem with non-JBoss products. The answer is always the same: "you have to know which interface you want to use". This isn't hacker level stuff - if you believe it _is_ at the hacker level, I guess you're targetting 13 year olds who have a day job at White Castle and are setting up JBoss clusters at night.

    \Bela Ban\
    What are you so concerned about *my* reputation ? Again, this transport is a *deployment* issue, you *don't* need to use it.
    \Bela Ban\

    Clearly, this wonderful protocol is going to be your FAQ for finding other servers - "oh, just use the easy protocol". And a whole bunch of JBoss clusters are going to get set up using it, because it's the path of least resistence. And you'll slowly morph into the Microsoft model - everything easy to use and fundamentally broken. Just like Microsoft server systems get regularly slammed for doing too much, opening up too many listeners that aren't needed, so are you. I'll continue to be amazed that you don't see how fundamentally ridiculous this idea is.

        -Mike
  42. Open Source vs Commercial Vendor[ Go to top ]

    "The only thing my post was trying to convey was that they've outlined a huge amount of work and apparently allocated only 6 months in which to do it all - in addition to other obligations. "

    Hey, that sounds like my employer!
  43. Open Source vs Commercial Vendor[ Go to top ]

    I think you lost the thread a bit here. The only thing my post was trying to convey was that they've outlined a huge amount of work and apparently allocated only 6 months in which to do it all - in addition to other obligations. Talk all you want about the benefits of open source, but it doesn't work miracles.


    Totally agree. In fact, the great thing about open source is that those projects usually do take the time to DO IT RIGHT. In fact, I don't think it's bad that IBM is ahead of them - with open source the code is ready when it's ready.

    Look at Linux. They has no problem letting the schedule slip because stability is number one.

    I don't think the JBoss business model makes much sense. To make money they need to get hired as consultants or train people. Now, of course, if they just opened up the docs like everybody else, they would lose that revenue stream, so just say consulting and training. Now, the cost of JBoss consultants must be pretty high (or else it's not worth it, no?). But typically if a manager has money to spend on consultants, then wouldn't he prefer to spend that money on a WebLogic or WebSphere license, given that those products have the most market share. They even have free documentation.
  44. Open Source vs Commercial Vendor[ Go to top ]

    But typically if a manager has money to spend on consultants, then wouldn't he prefer to spend that money on a WebLogic or WebSphere license, given that those products have the most market share. They even have free documentation.


    Sorry, but that's a silly statement. Let's spend money on a license that gives you absolutely nothing rather than spending it on a consultant that can jumpstart your development or training that can expand your company's knowledge. You could actually probably get both for the same cost as a license. You also talk like our documentation is $5000 per CPU. It is $30. We also have 2 free getting started books, one of which we actually contracted somebody to write.

    Bill
  45. Open Source vs Commercial Vendor[ Go to top ]


    > Sorry, but that's a silly statement. Let's spend money on a license that gives you absolutely nothing rather than spending it on a consultant that can jumpstart your development or training that can expand your company's knowledge. You could actually probably get both for the same cost as a license. You also talk like our documentation is $5000 per CPU. It is $30. We also have 2 free getting started books, one of which we actually contracted somebody to write.
    >
    > Bill

    Yeah you were right in san francisco, thought, that giving away the doc, now that we can afford it makes sense :)

    marcf
  46. Open Source vs Commercial Vendor[ Go to top ]

    Sorry, but that's a silly statement. Let's spend money on a license that gives you absolutely nothing rather than spending it on a consultant that can jumpstart your development or training that can expand your company's knowledge. You could actually probably get both for the same cost as a license.


    But since WebLogic/WebSphere are the market leaders wouldn't they go with the market leaders instead?

    > You also talk like our documentation is $5000 per CPU. It is $30. We also have 2 free getting started books, one of which we actually contracted somebody to write.

    I didn't mean to say that the documentation is expensive. But that reinforces my point that it probably doesn't count much as revenue (or does it?)
  47. Open Source vs Commercial Vendor[ Go to top ]

    jBoss inovated Hibreante, AOP, MicroKernal. What did Commercail Vendors do other than stay in a EJB heard?


    JBoss didn't innovate the above features. JRun was based on a microkernel architecture. JBoss is integrating Hibernate into its core product, but Hibernate is a persistence framework - it's independent (and rightly so) of any particular app server. Many vendors have implemented a persistence framework, including IBM's original Visual Age Persistence framework, which has been around for years. Of course, JBoss is unique in the scope of its ambitions for AOP, but commercial vendors (e.g. BEA, IBM) have made moves to incorporate AOP techniques into their products.

    I agree with your statements re: competition and choice, these are essential.
  48. Open Source vs Commercial Vendor[ Go to top ]

    <VIC>
     jBoss inovated Hibreante, AOP, MicroKernal. What did Commercail Vendors do other than stay in a EJB heard?
    </VIC>

    Pardon my language, but do you have any footing in the real world? Hibernate is Gavin Kings "invention", and I doubt hiring him once it was a success counts as "innovation", I´d say its basically good recruiting.

    AOP? You seem to forget about what academia and Xerox PARC and IBM (a few "commercail vendors" to use your language) have been working on since at least the mid 90:ies.. AspectJ comes to mind..

    Microkernel? Hardly a JBoss innovation..

    Common, most of us like JBoss, but trying to claim that a bunch of innovations that existed long before they came along where "their work" is just dillusional..
  49. JBOSS and innovation[ Go to top ]

    By innovation he obviously meant - being among first using all those technologies in J2EE application server, not actually creating them.
  50. JBOSS and innovation[ Go to top ]

    OK. But he's still wrong.
  51. Open Source vs Commercial Vendor[ Go to top ]

    Common, most of us like JBoss, but trying to claim that a bunch of innovations that existed long before they came along where "their work" is just dillusional..


    I think you mean "delusional" here?
  52. Open Source vs Commercial Vendor[ Go to top ]

    Pardon my language, but do you have any footing in the real world? Hibernate is Gavin Kings "invention", and I doubt hiring him once it was a success counts as "innovation", I´d say its basically good recruiting.

    >
    > AOP? You seem to forget about what academia and Xerox PARC and IBM (a few "commercail vendors" to use your language) have been working on since at least the mid 90:ies.. AspectJ comes to mind..
    >
    > Microkernel? Hardly a JBoss innovation..
    >
    > Common, most of us like JBoss, but trying to claim that a bunch of innovations that existed long before they came along where "their work" is just dillusional..
    >

    We do not claim we are inventing these. What is innovating is that we are the first to mass market these technologies. No one today except us excells or even has all of these technologies in commercially available products (we qualify as commercially available product :)

    BEA? give me a break, apart from the occasional rash of jealousy from cedric, there isn't much at BEA going on, they would have to rewrite their codebase to get the microkernel stuff right and keep up with us. Then they would have to make it free to compete with us in OEM, funny.

    IBM is ahead. And I consider them personal friends. But ahead at ***IBM RESEARCH*** and the work the folks at IBM Research are doing is remarkable (aspectJ, hyperJ etc). But IBM Websphere, the product? pffff off in "integration, schmintegration" land.

    JBoss IS an innovative bunch of hard core techies. We may not have invented any of the features we market, including group communication by bela ban in JGroups (JBoss inc staff) but at least we get it and promote innovation in java. Or would you rather discuss the fine points of configuration of web schmervices for EJB for another 3 years?

    marcf
  53. There are several things that stand out in this discussion.

    First (any maybe I'm just being negligent in my job), but why should I care about microkernels and other things related to implementation details. If you have a sophisticated product and you think there are differentiating features in it, then describe how it benefits me, the application designer, developer, and decision maker. If you keep talking in mumbo-jumbo, your message won't likely be heard by anyone who is a decision maker.

    Second, I've been looking for information or studies comparing open source and commercial software. What is cheaper overall? What produces better quality products? What is more robust? What is easier and faster to develop with? There don't appear to be many answers.

    You can claim I'd be better off spending my dollars on developers as opposed to licenses, but if I have half a brain I am going to look at total costs, among other things.

    Unfortunately, I am starting to conclude exactly opposite what many people in the open source community preach. I see Microsoft being better suited for the small to mid size business. The development tools are superior to anything out there for any platform (this is a fact, so don't bother disagreeing), and allow the fastest development. I can get basic functionality for web, database and ecommerce out of the box. Even a technical person with no background in MS technologies can get started quickly. In a business sense, when planning a project I can more easily forecast a budget and be accurate.

    Now look at it from the business perspective and that will reveal JBoss's problem in selling to big business. Many decision makers at big business want to buy something that is safe (low in risk to them and their jobs). There is a saying about, "nobody ever got fired for hiring IBM". That means perceptions (not reality) about support, quality, and costs exist.

    So maybe all this talk about JBoss vs commercial vendors is irrelevant. JBoss will appeal to the low hanging fruit - the students who can't affort to purchase BEA, the companies too cheap to spend any real dollars on development and will likely fail in the development efforts anyway. Or it will appeal to the mega companies who can afford the risk and have the resources to do a sound financial comparison (with ROI studies, etc...). In either case, neither place is good to be as a developer who wants to make a decent salary and work on interesting projects with open source.

    Mike
  54. There are several things that stand out in this discussion.

    >
    > First (any maybe I'm just being negligent in my job), but why should I care about microkernels and other things related to implementation details. If you have a sophisticated product and you think there are differentiating features in it, then describe how it benefits me, the application designer, developer, and decision maker. If you keep talking in mumbo-jumbo, your message won't likely be heard by anyone who is a decision maker.
    >
    Finally an interesting question! Why do you need a microkernel? I'll be brief, but I hope you get my point. J2EE has no concept of a service, initialization (let's call it lifecycle), or dependency management. For instance, one of our clients had a read-mostly site and needed various custom caches. One cache service had the requirement of reading a data feed every fifteen minutes for a preconfigured limited set of frequently used quotes. Using our microkernel they were able to define a service that interfaced with the datafeed and was initialized at startup(lifecycle), write the caching service and not have it start filling the cache until the datafeed service was initialized(dependency management), have a framework to easily preconfigure desired quotes to cache and period for refreshing the cache. Finally because these services were written to our microkernel, they instantly had GUI management through our JMX console without having to write a line of UI code so that they could dynamically invalidate the cache or change cached quotes with a click of a button. (Recently we've added the ability to graph JMX attributes, alert monitoring, and snapshot recording and graphing of attributes.) They can even remotely invoke on any of these services through a variety of protocols.

    Now how easy was it to write a service? All they had to do was write classes that implemented our service interface (start, stop, destroy, create) and to define a simple Java interface suffixed by "MBean" to define the management interface attributes and operations.

    Besides this example, I encourage you to read up on Inversion of Control as well as we have already incorporated and will incorporate man of the ideas of this design pattern into our kernel.

    Ok, I wasn't brief, but here's some more of why the microkernel is important. If you are an ISV, the microkernel provides a way for you to define any daemons you want running within our application server, or even a way for you to plugin your proprietary components into our runtime and gain the benefits of our lifecycle, dependency, management, classloading and hot deployment. The Hibernate project is a good example of somebody that will benefit from such a service.

    Bill
  55. Open Source vs Commercial Vendor[ Go to top ]

    [..] because these services were written to our microkernel, they instantly had GUI management through our JMX console without having to write a line of UI code so that they could dynamically invalidate the cache or change cached quotes with a click of a button.


    One thing I have wondered about in the past is: how do you deal with management on a cluster. Are you doing a two-phase commit on your component replicas using javagroups?

    For example, when you deploy components on a weblogic cluster it appears to do a two-phase commit (but no database, obviously).

    Do you do this with your management GUI?
  56. Open Source vs Commercial Vendor[ Go to top ]

    Two minor points of consideration:

      1) There's nothing "micro" about your implementation. What you have is an infrastructure for initializing and accessing services. It's not a micro-kernel by any stretch of the imagination. As I said before - there are too many dependencies to be tagged as "micro".

      2) Big deal - every container I've ever heard of has service init and access facilities. This isn't a differentiator for a container, it's a bare minimum requirement. It's true that there is no J2EE-standard for it, and as such you need to use a vendor-supplied mechanism to do it. Fortunately, every single vendor supplies one ;-) That said, it will be nice when J2EE actually does standardize this better (and I believe it's moving in that direction).

    As I mentioned before - take J2EE out of the picture, and your competition is no longer just Websphere and Weblogic and Jonas and someday Geronimo (all of which have this new-fangled "services" concept, BTW). Your competition is now also Pico and Nano and even (heaven forbid) Spring, just for starters.

         -Mike
  57. Open Source vs Commercial Vendor[ Go to top ]


    > Besides this example, I encourage you to read up on Inversion of Control
    >
    > Bill

    You guys have been dropping hints about IOC for months, marcf cited a need for it a while back. It's odd not to see it in your road map. If you already know that the Spring – Nanning – CGLib approach will support fast lightweight EJB aspects why not be more forthcoming? It would alleviate skepticism about JBoss 4, especially if you don't roll your own IOC framework.

    Gary
  58. Open Source vs Commercial Vendor[ Go to top ]


    > > Besides this example, I encourage you to read up on Inversion of Control
    > >
    > > Bill
    >
    > You guys have been dropping hints about IOC for months, marcf cited a need for it a while back. It's odd not to see it in your road map. If you already know that the Spring – Nanning – CGLib approach will support fast lightweight EJB aspects why not be more forthcoming? It would alleviate skepticism about JBoss 4, especially if you don't roll your own IOC framework.
    >
    > Gary

    The point is that for many things that IOC and IOC containers are being used for we already support in our microkernel. There are some bullet points in the roadmap to improve our microkernel and package it as a separate download (MicroJBoss).

    Bill
  59. \Bill Burke\
    The point is that for many things that IOC and IOC containers are being used for we already support in our microkernel. There are some bullet points in the roadmap to improve our microkernel and package it as a separate download (MicroJBoss).
    \Bill Burke\

    Well, you have MBeans and JMX and XML files and a home grown AOP system.

    Oooh, and a classloader, too!

    Mix them all together and that's just-like-IOC. Just like your aging J2EE interceptor stack was just-like-AOP. As with the latter, they've been doing the former all along!

    Other things which JBoss will soon announce they've been doing all along:

       - Ruby. "We have a 3 line Java interface that works just like a Ruby closure - see, we've been doing Ruby all along!".

       - RDBMS. We store some stuff persistently and let you get read it back in, just like an RDBMS! Oracle sells their system for a gazillion dollars per byte of RAM, but our JPersistinator is free! And when we hire that Prevayler dude we'll own the whole market!

       - Perl. We too have 57 different ways to do everything. So logically we've been doing perl all along - even longer than perl has.

       - Grid computing. Marc copied a class from a web site 3 years ago called Grid.java, so naturally we've been doing grid computing for years without even realizing it!

       - Design by Contract. Marc made us sign a contract that insists that all of our designs have to be 1 paragraph or less in length, and use the phrases "mind blowing" and "Jboss Jedis" at least 3 times. We're confident that JBoss is the undisputed king of Design by Contract!!!

       - Highly fault tolerant and recoverable computing. Oh, no wait, we don't even have a transaction log. We'd better skip that one....

       - Massively parallel processing. We've done extensive surveys, and we've found the 13 JBoss servers used in production today waste 17 times more CPU power than all the SETI screen savers combined.

       - AI. Half of our code is run by a random number generator. Hey, we invented fuzzy logic! All those Prolog people are a bunch of academic wankers, we've made AI a commercial reality with our random-path Enterprise Transaction software.

       - Robust 2PC architectures. Bill Burke swears our web site is running JBoss on 2 PCs, and they haven't melted down to slag yet. You may have heard from some crocodile that "2PC" means Two Phase Commit, but those are just whiners who want to steal some of our glory. Only JBoss can run on two honest ot God personal computers (and those crazy academcians can just SMD!). You see, JBoss has been running on 2 PCs since 2003 (when we could finally afford the second one). In 2004 we'll be announcing our 4PC architecture using advanced SMP techniques (for those not in the know, SMP is the more biologically correct phaseology for SMD. I didn't know they call such a personal act "architecture", but if it makes me money, I'm willing to sacrifice!!!).

       - Algorithms. Marc-the-math-major found a book that has these al gore rhythms in it (psst, it's Algorithms Marc, forget Al Gore!). Extensive analsysis of the JBoss code base has found that JBoss has been using algorithms since day one. Only JBoss uses Algorithms - all those commercial money grubbers like IBM and BEA can't use algorithm because they're closed source. If you want to start using Algorithmic computing, you know who to turn to!

         -Mike
  60. Open Source vs Commercial Vendor[ Go to top ]


    You guys have been dropping hints about IOC for months, marcf cited a need for it a while back. It's odd not to see it in your road map.
    >

    You are correct, we should articulate that part. However did I mention IoC is gay?

    seriously. We have been talking about it. I partied with Haslak and jon (of pico fame) in antwerp after the zorro trick, the grass in antwerp is very good :).

    The point is that after thinking long about it there is really one feature about IoC, that feature is "inversion of control" :) or more precisely "control of references by the system". So you get a container, and the main thing it does is set the references transparently on your objects. So your object graph LINKS are under control of the system. Most of the benefits of IoC come from this simple feature (for example independence of lookup mechanisms). That is a good thing. Not ground breaking, not "aaaaaaaah, I see the virgin" good but a good thing, system control is always a good thing :).

    One place where I would find it useful (unlike lookup independence which i find gay) is control of cache structures of SESSIONS that means runtime control of the references not just "set up/lookup". EJB's implement that behavior with session and passivation (EJB implement runtime IoC in fact). We have a need today in the generalized containers vision for control of session cache structures. So we can passivate and move sessions around by system control. This is properly done in HTTPSession replication in JBossTomcat, the tomcat clusters on JBoss.

    Once the system controls the references in a graph it can implement semantics for it (time to live, caching, creation, passivation bla bla bla). It isn't a standalone framework though it is a nice feature of frameworks.

    marcf
  61. Open Source vs Commercial Vendor[ Go to top ]

    Not ground breaking, not "aaaaaaaah, I see the virgin" good but a good thing,

    > system control is always a good thing :).

    Sheesh, and I thought you were a red-pill kind of guy. Not only do you prefer grass, you like system control. Well whaddyaknow.

    All kid-ding (get it? and is it?) aside I'd say the problem you describe is probably best solved by introducing another layer of indirection, preferably by using dynamic proxies. Can't beat that trick, now can you.

    /R
  62. Open Source vs Commercial Vendor[ Go to top ]

    I'd say the problem you describe is probably best solved by introducing another > layer of indirection, preferably by using dynamic proxies. Can't beat that

    > trick, now can you.

    Yes you can. But you have to live in the runtime level, not at the Java language level. But then this is where this added indirection really belongs to. Below the language I use to write my applications.

    /T
  63. It's called InetD[ Go to top ]

    Lots of rants from many side...lots of marketing bullshit from all sides...lots of ahem...inexpertise from all sides.

    Microkernel -> inetd or look up OSGi service gateway which has been around for at least 3-4 years...

    You guys are like frogs in a deep well looking up and declare that little circle of sky you see above is the Universe...

    :-)
  64. Meant for everyone :-)[ Go to top ]

    Btw, my comments were directed at Everyone :-)
  65. It's called InetD[ Go to top ]

    Lots of rants from many side...lots of marketing bullshit from all sides...lots of ahem...inexpertise from all sides.

    >
    > Microkernel -> inetd or look up OSGi service gateway which has been around for at least 3-4 years...
    >

    I am no linux guru BUT

    JBoss microkernel == Linux insmod

    insmod or (insert module) is the capacity to dynamically add modules or services to a running kernel. Heck that qualifies as a microkernel in my book. inetd is just a RPC mechanism to call the services that listen to ports, inetd == detached invokers in JBoss (xinetd is even better in that it launches as needed).

    marcf

    PS: inexpertise????
  66. Open Source vs Commercial Vendor[ Go to top ]

    (Recently we've added the ability to graph JMX attributes, alert monitoring,

    >and snapshot recording and graphing of attributes.) They can even remotely
    >invoke on any of these services through a variety of protocols.
    What do you mean here and how to see this feature?
    Currently we are using JConsole (www.servletsuite.com/jmx/jconsole.htm) instead of your default management

    Thank you,
     Eugene
  67. Open Source vs Commercial Vendor[ Go to top ]

    (Recently we've added the ability to graph JMX attributes, alert monitoring,

    > >and snapshot recording and graphing of attributes.) They can even remotely
    > >invoke on any of these services through a variety of protocols.
    > What do you mean here and how to see this feature?
    > Currently we are using JConsole (www.servletsuite.com/jmx/jconsole.htm) instead of your default management
    >
    > Thank you,
    > Eugene

    you need the JDK 1.4 plugin for your browser. Then go to /web-console.

    In the JMX tree, find an mbean and expand its attributes. Right click on attribute and you should get a popup menu to graph, or monitor an attribute. Snapshot Recording will be in the 3.2.4 release.

    Bill
  68. Open Source vs Commercial Vendor[ Go to top ]

    We do not claim we are inventing these.

    Not saying that, plainly just reacting to Vics somewhat misdirected words.

    <marcf>
    JBoss IS an innovative bunch of hard core techies. We may not have invented any of the features we market, including group communication by bela ban in JGroups (JBoss inc staff) but at least we get it and promote innovation in java.
    </marcf>

    Yep, and I think most of us are thankful for it. Being innovative can be using known quantities in new ways. I am not familiar with your AOP plans yet, but being a fan of AOP (and AspectJ so far), I can only see good coming from you guys trying to make it more "mainstream".
  69. Open Source vs Commercial Vendor[ Go to top ]

    We do not claim we are inventing these.

    > Not saying that, plainly just reacting to Vics somewhat misdirected words.
    >
    > <marcf>
    > JBoss IS an innovative bunch of hard core techies. We may not have invented any of the features we market, including group communication by bela ban in JGroups (JBoss inc staff) but at least we get it and promote innovation in java.
    > </marcf>
    >
    > Yep, and I think most of us are thankful for it. Being innovative can be using known quantities in new ways. I am not familiar with your AOP plans yet, but being a fan of AOP (and AspectJ so far), I can only see good coming from you guys trying to make it more "mainstream".


    Wille,

    DUDE, 110 posts later and ONE posts says "thanks keep it up". Pheeewww, what a saturday, at least I am off the hook and I can go practice my figure skating jumps

    marcf
  70. Impressive?[ Go to top ]

    So, they do have "several man-years" or at least can begin to approach it.


    The "large community" thing...


    The formula is a bit different than what you say though the man years are irrelevant because

    1 average joe year == 1 mythical man month

    so reading the roadmap with regular eyes ignores how "PROFESSIONAL OPEN SOURCE" works. It works with a COMBO of pros in professional open source and amateurs in open source.

    Most of the guys on board at jboss inc have proved their worth in open source. Meaning they are self starter, capable of working remotely, self motivated bla bla bla. The bottom line is that writting an advanced product requires talent first and then time. I SCOUT THE EARTH in search of that talent. Think Bill, Sacha, Remy (tomcat), Gavin (hibernate), Bela (JGroups) etc etc. Then the effect of the community is demultiplied. Meaning we tend to attract a great crowd of talent. The reviewer of JBoss code are usually corporate high-fliers, the go to guys, the "real" lead architects (not the clowns that draw boxes on white boards and then get rashes when they look at actual code), so the feedback that we have in Q/A and new features is usually of great quality. The Q/A of open source is UNIQUE (when it works) not even large corporations can afford that amount of quality feedback.

    BOTTOM LINE:

    Professional Open Source works with a small core of "pros" and a large community of help in q/a additional features.

    marcf
  71. How come JBOSS never wants to address the issue of improving their documentation ? I m sure with great documentation a lot of users specially starters will go for JBOSS than say RESIN etc. Why JBOSS team thinks documentation is not a part of their strategy or roadmap or whatever u may want to call it.
  72. I agree, JBoss's documentation sucks, I've been trying to get my hands on JBoss 3.0 and it has been very frustrating. I know the software is a piece of art but without the proper documentation won't have the impact that could have.

    Take a look at JOnAS proyect. Very robust and compliant J2EE Open Source App Server. The documentation is far superior and the page is better organized.

    http://jonas.objectweb.org/doc/index.html

    Rodrigo
  73. Documentation?[ Go to top ]

    Rodrigo,

    Are you talking abouth the documentation they sell?

    Johan Withag
    jwithag@withag.com
  74. Jboss must be a Learning Center[ Go to top ]

    I dont know, are the Geronimo, Objectweb, or any J2EE compliant will support this? If yes, say goodbye to JBoss.

    I believe that JBoss will become the most respected solution if make more detail documentation, easy to use. and learn from Microsoft what are they do.

    I dont know if they pushing us to buy their training and support, that is another startegy. :) but believe me more cheaper more popular.
  75. Jboss must be a Learning Center[ Go to top ]

    I believe that JBoss will become the most respected solution if make more detail documentation, easy to use. and learn from Microsoft what are they do.

    >
    > I dont know if they pushing us to buy their training and support, that is another startegy. :) but believe me more cheaper more popular.
    >

    "more cheaper, more popular" we are alreday there wouldn't say? we are free and mass distribution #1 in development, #1 in OEM, #1 in ISV.

    you are correct though, training people on our stuff is the most important, microsoft is a great example to follow indeed. Giving away our docs will make sense soon.

    (BTW if you are an academic doing relevant research, talk to us about training, we will surprise you ;)

    marcf
  76. The documentation situation[ Go to top ]

    Ah yes,

    the documentation situation.

    1- If you read it closely the FOR PAY documentation is really good, it is very detailed, thorough etc, we even refer to it among ourselves at JBoss inc. What used to suck was the FREE documentation, the one where the "community" could help. What a fucking joke "free communities" are, at the end of the day it is a small group of people that do all the work and money works too. We let the documentation there for a couple of semesters and IT DIDN"T MOVE. So I decided to invest a good 10k in the FREE doco from a writer and certified JBoss consultant the result has been online for a couple of weeks, try it out.

    2- We have been putting it for pay for a while and that in fact got the company off the ground back in the days it paid for my partner, Scott Stark, salary. Today the revenues are way more significant than the revenue doco provides and may very well give it away in the near future. Read again 80% probability we give ALL documentation away (someone talked about more mass more penetration we are there). The only problem there is that the information gathering from doco is important and I still don't want to ask for information at code downloading so we got a find a way to collect info at doco download etc.

    3- When we do we will follow the mySQL model (**** jonas and all these french govt clowns :) where it is online and we invite wiki feedback style for all of it so that regular users can improve on it.

    4- Remember what is important is the quality of a product, quality of doco will follow given the proper resources (== money). We have trained thousands and thousands of developers, newbies and jedis, with this documentation. The product is the probably the best on the market, any jedi will tell you, I am telling you :)

    problem solved soon

    marcf
  77. The documentation situation[ Go to top ]

    Ah, I guess the kinder and gentler Marc Fleury experiment has failed, and your resolution for 2004 is to go back to your old self, eh? In any case, thanks for the treasure trove of useful information...

    1) Free communities are a fucking joke. I'm sure all of the non-JBoss Group people who have contributed to JBoss will just _love_ you for that sound bite.

    2) Your closest open source J2EE competition are french clowns. I suppose once Geronimo achieves critical mass you'll have to invent a new epiphet for them (CDN jackoffs? No, not offensive enough for you...).

    3) Documentation is a secondary concern. Frankly, IBM and BEA and others will eat your lunch with that attitude. The best code in the world is worthless if you don't understand how to use it - and sorry, no, the majority of potential users in the world are _not_ going to go to JBoss Group for training.

    4) Ah, lots of uses of the "jedi" phraseology. Wow, anyone want to guess who "code freedom" really is? Come on, go way out on a limb....I guess code freedom and race condition and j2ee master and several others can get together and have a great big party some day. Of course you and Bill Burke might get rather confused trying to figure out which name is "in character" at the moment, but throw in some heavy drinking and it might be fun...."guess which TheServerSide psuedonum I'm using now, Bill.....".

        -Mike
  78. The documentation situation[ Go to top ]

    Ah, I guess the kinder and gentler Marc Fleury experiment has failed, and your resolution for 2004 is to go back to your old self, eh? In any case, thanks for the treasure trove of useful information...

    >
    > 1) Free communities are a fucking joke. I'm sure all of the non-JBoss Group people who have contributed to JBoss will just _love_ you for that sound bite.
    >

    What Marc is refering to is that free communities usually want to contribute to the fun, interesting work, but shy away from the grunt work like documentation, regression tests, UIs, and other not-so-sexy work. You need a synergy between the commercial world and the ideals of the FSF. I think we've found this with our Professional Open Source model.

    Bill
  79. The documentation situation[ Go to top ]

    What Marc is refering to is that free communities usually want to contribute to the fun, interesting work, but shy away from the grunt work like documentation, regression tests, UIs, and other not-so-sexy work. You need a synergy between the commercial world and the ideals of the FSF. I think we've found this with our Professional Open Source model.


    So how are you going to get all these fun-loving unpaid developers to do all the QA work for you without being part in the actual fun part of designing and building the features?

    R
  80. The documentation situation[ Go to top ]

    What Marc is refering to is that free communities usually want to contribute to the fun, interesting work, but shy away from the grunt work like documentation, regression tests, UIs, and other not-so-sexy work. You need a synergy between the commercial world and the ideals of the FSF. I think we've found this with our Professional Open Source model.

    >
    > So how are you going to get all these fun-loving unpaid developers to do all the QA work for you without being part in the actual fun part of designing and building the features?
    >
    > R

    Because users of JBoss tend to do QA because they find the bugs during development of their app, and fix the bug. Because one of the things to get involved coding JBoss is fixing a few bugs so that we know that you know sort-of what you are doing and have initiative. I was a user 3 years back and this is how I got involved with JBoss.

    This is the QA open-source advocates talk about. Users fix their own bugs(manytimes).

    Bill
  81. The documentation situation[ Go to top ]

    2- We have been putting it for pay for a while and that in fact got the company off the ground back in the days it paid for my partner, Scott Stark, salary.


    Not only Scott, but myself, Sacha, and even Dain received good money from doco sales to supplement our income.


    > 3- When we do we will follow the mySQL model (**** jonas and all these french govt clowns :) where it is online and we invite wiki feedback style for all of it so that regular users can improve on it.
    >

    The wiki idea can easily become reality with Nukes and the new forums written on Nukes. Julien Viet has written some cool stuff.

    > 4- Remember what is important is the quality of a product, quality of doco will follow given the proper resources (== money).

    LOL. The open source model definately doesn't work with documentation. Marc is right. Our documentation situation was much much worse when we relied on volunteers. At least now we have quality documentation even if you have to pay a measly $30 bucks.

    Bill
  82. Roadmap to the Wrong Place[ Go to top ]

    IMHO it's useless to write an open-source implementation of J2EE. The purpose of specs, like Java, J2EE etc. is to manage risk. On the other hand, the purpose of open-source is also to manage risk. If I use a spec-compliant product, I can always replace it with another one. If I use an open-source product, the likely that I'll need to replace it is almost nil.

    So, specs and open-source are really an either/or problem. You can have both (JBoss), but what's the point? Specs can be good, but usually they are not that great, and often they are REALLY bad (and politically motivated.) Just look at EJB 1.1.

    Just look at how well Hibernate is doing. There's no committee of big names behind it, but it's doing great. On the other hand, if it was closed source, then it would look very risky and nobody would use it.
  83. Huh?[ Go to top ]

    Not all companies use opensource technologies. You are tying yourself to a "product", not a technology. Hibernate is a product. JDO is a spec. The fact that the spec may not be the best is beside the point.
  84. Huh?[ Go to top ]

    Not all companies use opensource technologies. You are tying yourself to a "product", not a technology. Hibernate is a product. JDO is a spec. The fact that the spec may not be the best is beside the point.


    It depends on what you think the point is. As I said, to me the point is risk management. If the point is interoperability with other companies (as in web services), then, yes, the spec *is* the most important thing.

    But most people use standards as a way of hedging risks of various kinds, like vendor lock-in (high costs), the availability of a good labor pool, etc.

    Frankly, if JBoss replaced EJBs with their own POJOs (which they already have but EJB is still the main focus), they could still support web services.
  85. Huh?[ Go to top ]

    Not all companies use opensource technologies. You are tying yourself to a "product", not a technology. Hibernate is a product. JDO is a spec. The fact that the spec may not be the best is beside the point.

    >

    The point of specs as you mention is for business to feel safe. They can go with a free solution like JBoss but if they want they can go back to a for-pay software crap (in reply to the SUN "free software crap"). It is the way VP's think, minimizing risk is king, but that logic takes on a strange twist in the Free software world (see the "smile, you are stuck in a free world, it is good" post above).

    So our support of EJB vs POJO's is just doing what our users want, we got users that want both. I should also say that there is a STRONG TECHNOLOGY ***BELIEF*** in JBoss inc that pojo middleware is the right form of middleware, independently of standards and customers and bla bla. Yes it is 50% users, 50% tech for the sake of tech LOL.

    back to users.

    1- IT users. They want mostly J2EE compliant. Has nothing to do with the technology or the quality of the spec but mostly putting your company on a open standard. Today that means J2EE for java, in the future it may mean open source for java, already some signs show it. We are not there yet (a world where Free Software == the safe choice).

    2- ISV users. JBoss has a reported #1 MARKET SHARE IN OEM/ISV. It isn't very surprising as JBoss is free and modular and the service is excellent. So we are pretty much the only choice today. Plus our license, the LGPL assures the OEM world that we are not going to pull a BSD trick and make our product derivative closed. JBoss will ALWAYS be free BY LICENSE. THESE USERS I ARGUE *****DON'T CARE ABOUT J2EE/STANDARDS***** They care and can care about a FRAMEWORK, a container base to do their work. What minimizes my time in development and gives me what I need, is the question they ask. Pojo is closer to their answer than J2EE. We are seeing the uptake of AOP services in this ISV community. I strongly recommend you read the excellent article by BILL BURKE in the current JDJ. Bill did a wonderful job to show how to use the stuff TODAY and it solves ISV OEM problems in a simple fashion, way simpler than J2EE. Adoption is there, in the ISV community.

    3- SI. System integrators are for-hire and will do whatever the client wants. "For the right pay, sure honey!". Governments have been asking for OPEN SOURCE before they ask for standards, and these clients mean mucho dolares. The govt want JBoss (US, Europe) we partner with SIs in these cases, in the JASP program (JBoss Authorized Service Partner). Many governments ask for open source AND standard for their infrastructure parts.

    The dynamic of adoption is strange and a mix of standard, risk, marketing, technical edge and random luck. Right now you find a complete J2EE solution in JBoss (top quality too) and a visionary push on technology, middleware for pojos.

    marcf
  86. Marc, your propaganda's getting a bit tattered from all the recycling. You might want to freshen it up a bit.

    1) A good standard always beats all other comers - even if they're open source. The problem to date has been that J2EE has had some big holes in it. Those holes are being filled, and as that happens you'll find more proprietary open source solutions fall to the way side.

    2) BSD and LGPL. Tens of thousands of lines of code have been transferred from the LGPL to BSD-style licenses this year, much of it as part of the on-going Geronimo effort. Even pieces of JBoss have been legally re-issued under the Apache license. In the end, you're doubly wrong here: first, code issued under BSD, Apache, or LGPL remains free in perpetuity. It's there, take it. Someone copying it and modifying it is a new case, new software - those changes may or may not remain under the original license - and who cares? What's free now is still free. Second, obviously LGPL code can be transferred to a BSD license. It's happened, it's out there, and it's legal. Rant all you want, but your assertions about LGPL guarantees are flat _wrong_.

    3) You have _lots_ of competition, and they erode your market share on a daily basis. You're the biggest rooster who crows the loudest, but other open source projects are quietly chewing into your market. As you piss off more and more of the development community, expect that trend to continue to accelerate.

    4) AOP is so vitally important to your vision that nobody's touched any of the AOP code or "aspects" since July. The cache project is so important that Bela's off playing with auto-configuration stuff in JGroups in preference to coding for you.

    AOP is so important that you're doing all the J2EE 1.4 work _first_, with POJO AOP a hazy dream as part of some "long term plan" with not even a weak timeline attached to it.

    5) Take away J2EE, and your entire market collapses. W/out a standard to latch onto, the IoC containers and individual component makers and other open source projects you haven't bought out will eat your lunch handily - and are starting to now. Take away J2EE and people say "why the hell do I need this big honking JBoss thing? There are plenty of lighter weight containers out there that get the job done _today_, and as a bonus the principals don't curse at me every third day and twice on Sunday.

    6) Most active open source projects are highly responsive to bug reports, and the CVS activity reflects this. JBoss is increasingly acting like a lumbering dinosaur, and you're lucky if someone responds to a forum question 3 months after you posted it. Many key developers have quit the project outright, the remainder either have their own open source wares to maintain, or are out on billing, or both. Whether you know it or not, you're rapidly Sybase-izing yourself - you've lost your focus and momentum, your core claim to fame is withering on the vine, and your competitors are streaking past you at an astonishing rate. You've failed to learn the lessons of history: neglect your core strengths, flail around too much in new ventures, and you'll find out how _really_ important that core was - after it's atrophied to the point of uselessness, unfortunately. The path you're taking now is the Sybase path - the core raison d'etre of JBoss is its J2EE container, you've let development on that fall way off in persuit of other ventures, and _that's_ what people will notice the most. Keep it up and it won't surprise me if 2 years from now JBoss isn't even mentioned for new projects, and only poor suckers who are locked-in to it still use it (like many in the financial community with Sybase).

         -Mike
  87. Keep it up and it won't surprise me if 2 years from now JBoss isn't even mentioned for new projects, and only poor suckers who are locked-in to it still use it (like many in the financial community with Sybase).
    >
    >      -Mike

    Mike,

    Can you please elaborate a little bit on the Sybase stuff? You probably mean entering in collision with diverse big players hadon(PB against VB, ASE against Oracle and MSSQLServer).
  88. \Double DODO\
    Can you please elaborate a little bit on the Sybase stuff? You probably mean entering in collision with diverse big players hadon(PB against VB, ASE against Oracle and MSSQLServer).
    \Double DODO\

    Sybase's core competency in the 90's was databases - the engine inside of their RDBMS system. It was state of the art, was in all the benchmarks, was seriously considered for new purchases, was a big-time "player" in the field. Somewhere around Sybase (10|11, can't remember exactly), Sybase went on a big expansion binge. They were branching out into all sorts of endeavours, coming out with all sorts of products in diverse categories - and along the way to doing that, they started neglecting the core database engine. They kept up basic maintenance stuff, but didn't aggressively push the envelope in the database for features, robustness, speed, scalability, etc. They came out with (10|11), a big yawn of a release from many respects - and that's exactly what the purchasers of RDBMS systems did. They yawned.

    While Sybase was frantically spinning out products left and right in all sorts of fields, and letting their RDBMS development slide down the crapper, Oracle and Microsoft were really hitting their stride (ironically, Microsoft was doing it with code that they bought originally from Sybase). While both of those companies obviously were branched into all sorts of things, they _also_ kept a keen focus on their database properties. SQLServer and Oracle kept banging out RDBMS releases with bold new features, more fault tolerance options, better speed, better scalability. They took the market seriously.

    The result? Sybase's RDBMS offering took a huge plunge in the market. They fell increasingly behind Oracle and Microsoft. Over an astonishingly short time span, they basically lost the market completely. Along the way, they found that the RDBMS was their philosophical and financial core - RDBMS _was_ the company, and it was the cash cow too. The company lost all credability in the RDBMS market, this flowed to their other product bases, and at the same time revenue plunged. Today Sybase is a hollow shell of who they used to be.

    IMHO, JBoss is making the same mistake. They're neglecting their core competency and thrashing about in other areas, and they're forgetting their business is based fundamentally on this J2EE container code. While they're neglecting this core engine, BEA and IBM are _very_ seriously focused on their app server offerings. On the other end of the spectrum, Jonas and Geronimo are also _very_ seriously focused on their code. And there's plenty of open source players in niches outside of that.

        -Mike
  89. Core competency[ Go to top ]

    Our core competency is middleware. I've lived and worked through the birth,life, and death of DCE and CORBA (at DCE and CORBA companies) and am seeing the signs that J2EE is entering its twilight as well. When you start seeing specs become bloated and tailored to individual vendors, or take years (J2EE 1.4) to come out with a final release, see open source and proprietary products come out to fill out the numerable holes in the spec, then you know the death is near. The only thing slowing the death of J2EE is a bad economy.

    Our microkernel is a strong implementation to rival even the other containers you talk about. We handle component depencies, life cycle, configuration, classloading, hot deployment, and have robust connector architecture to provide remoteness, security and transactions, and finally have an abstraction implementation so that developers can write plugins to deploy their own types of components and get automatically the features I described above. What we haven't done is package and productize our microkernel. There's also a tiny bit of work to do to make it more POJO friendly as you've seen in the 4.0 roadmap.

    We see Aspect-Oriented Middleware as the future. POJO based development is the future, gives you the highest flexiblity, and allows you to isolate business logic from middleware. To us, AOM, is mostly a repackaging of most of our existing technology. AOP is intuitive to us as we've been using aspect oriented techiques when we didn't know it was called AOP. J2EE a la carte via AOP, is taken directly and easily from our existing code base. AOP Remoting and AOP clustering, built apon the services already present in our 3.2 and 4.0 codebases, caching(Bela's stuff), although new, we are applying it in a pluggable fashion to our J2EE container in the CMP and SFSB arenas. We're also converting some existing interceptors in our J2EE implementation to aspects(i.e. transactional synchronization/locking). Finally, Hibernate completes our POJO strategy by providing a robust, flexible, and powerful persistence mechanism. These guys are so good, they were invited to the JDO 2.0 expert committee.

    Our core compentency, again, is middleware. Our project/product offerings at JBoss.org all complement and build off of each other.

    It's sad that all we get from this JBoss 4.0 roadmap on TSS is non-constructive babble Why not ask some constructive questions like "HOw does MicroJBoss compare to other lightweight containers and IoC?" "What will it take to implement JSR-88?" "What exactly has been implemented for web services so far?"

    Bill
  90. Documentation[ Go to top ]

    Without documentation this product is useless.

    I moved my project off Jboss because there are no good books available. No good documentation available.

    This product is use-less.


    Umesh
  91. Documentation[ Go to top ]

    It's very unfortunate that you got off of JBoss. We are pretty happy with our conversion from BEA to JBoss. About a month ago we unplugged our final URL from BEA/Portal and brought it up on JBoss.

    Here's a valid question for the JBoss guys: JB3 was so JCA-centric. I've seen Bill Burke mention the ... lack of simplicity of JCA. To say the least, we agree. Is JB4 moving away from the high dependence on JCA for its infrastructure (connection pooling, etc)?

    Steve
  92. Documentation[ Go to top ]

    It's very unfortunate that you got off of JBoss. We are pretty happy with our conversion from BEA to JBoss. About a month ago we unplugged our final URL from BEA/Portal and brought it up on JBoss.

    >
    > Here's a valid question for the JBoss guys: JB3 was so JCA-centric. I've seen Bill Burke mention the ... lack of simplicity of JCA. To say the least, we agree. Is JB4 moving away from the high dependence on JCA for its infrastructure (connection pooling, etc)?
    >
    > Steve

    Our only dependence for JCA is on JDBC connection pooling and J2EE 1.4 requires tighter integration with JCA. We have no intention of rewriting our connection pooling at this point.

    Bill
  93. Documentation[ Go to top ]

    It's very unfortunate that you got off of JBoss. We are pretty happy with our conversion from BEA to JBoss. About a month ago we unplugged our final URL from BEA/Portal and brought it up on JBoss.

    >
    > Here's a valid question for the JBoss guys: JB3 was so JCA-centric. I've seen Bill Burke mention the ... lack of simplicity of JCA. To say the least, we agree. Is JB4 moving away from the high dependence on JCA for its infrastructure (connection pooling, etc)?
    >

    be more precise, do you mean the connection pooling configuration?

    marcf
  94. Documentation[ Go to top ]

    Yeah. We got bit by JCA as we were trying to do some connection pooling in JBoss. We think we've figured out where we went wrong (not knowing enough about JCA in the first place) but I think it's a pain that has a better solution. Maybe there's another way to plug into the pooling mechanism.

    Steve
  95. Documentation[ Go to top ]

    Yeah. We got bit by JCA as we were trying to do some connection pooling in JBoss. We think we've figured out where we went wrong (not knowing enough about JCA in the first place) but I think it's a pain that has a better solution.



    Hear hear on that "JCA is overengineered". I think it is a perfect example of a spec that could fit in oh maybe 23 pages and is today a pompous bible. What are you going to do, each spec wants to appear real :)

    marcf
  96. Core competency[ Go to top ]

    Our core competency is middleware.


    yup, we own the OEM market and that direction is critical.

    > Our microkernel is a strong implementation to rival even the other containers you talk about. We handle component depencies, life cycle, configuration, classloading, hot deployment, and have robust connector architecture to provide remoteness, security and transactions, and finally have an abstraction implementation so that developers can write plugins to deploy their own types of components and get automatically the features I described above.
    >
    > We see Aspect-Oriented Middleware as the future.

    hear hear. It is funny, when you are on a don quijote crusade like we are with technology. You believe in it so bad, you want to assume the majority does as well. I think most of the guys here get it. The roadmap was a great effort by ivelin in making the focus clear, j2ee 1.4 compliance and AOP goal line. With goals we can hold our feet to (wait I am the CEO of the company...:).

    I think it is going to take a miracle for the MASS to get it. Maybe the right way to market is talk about the aspect USE right here, to a limited and smart crowd. The TSS crowd is what gets the word around on good technology.

    >
    AOP is intuitive to us as we've been using aspect oriented techiques when we didn't know it was called AOP. J2EE a la carte via AOP, is taken directly and easily from our existing code base.
    >

    yes, that is the difference in how we market AOP. AOP in itself is boring, pointcut languages and interception/weaving is the least of it. I was glad to be on the selection commitee of the AOSD this year and got to see all the research done by academics. They wanted a less snobbish conference. I was still amazed at the number of quality papers we rejected.

    I was pushing an industry agenda, that we should focus on the ASPECTS themselves, the use case of the stuff. The fact that you can cluster easily, by clustering invokers and state (cache). That you can have acidity in a distributed fashion with a high degree of configurability. That is what is important as you are saying bill. That you can have transparent persistence (hibernate IS an AOP impl with bytecode manipulation yada yada).

    I was thrilled to see RESEARCH where Quality of Service is seen as an aspect that can be bolt onto existing objects, orthogonaly. I was thrilled to see STREAM based semantics for remote calls, just like we already enhanced RMI to support pass by copy restore semantics (== pass by reference semantics) and having that as a CONFIGURATION OPTION on your SEMANTICS PALETTE. That is the kind of stuff that JBoss was made for and the Georgia Tech guys moved their implementation to JBoss in a day.

    > It's sad that all we get from this JBoss 4.0 roadmap on TSS is non-constructive babble Why not ask some constructive questions like "HOw does MicroJBoss compare to other lightweight containers and IoC?" "What will it take to implement JSR-88?" "What exactly has been implemented for web services so far?"
    >

    Put it to bed, let it be. TSS is a megaphone, not a "discussion forum". Discussion is on our forums at jboss.org where we discuss actual implementation of the stuff. Here the right people read about your stuff, you reach them with announcements. Most posts are good but some is 'crocodile' stuff (big mouths, little hands). Ignore the crocodiles lurking at TSS you are still reaching the crowd. Like juha once told me, it easier to talk trash than actually be building stuff.

    > Bill

    love you,

    marcf
  97. Core competency[ Go to top ]

    Bill


    >love you,

    >marcf

    I knew you guys were too close .... :)
  98. Core competency[ Go to top ]

    \MF\
    Put it to bed, let it be. TSS is a megaphone, not a "discussion forum". Discussion is on our forums at jboss.org where we discuss actual implementation of the stuff. Here the right people read about your stuff, you reach them with announcements. Most posts are good but some is 'crocodile' stuff (big mouths, little hands). Ignore the crocodiles lurking at TSS you are still reaching the crowd. Like juha once told me, it easier to talk trash than actually be building stuff.
    \MF\

    Marc, you might want to read your own forums a bit more often. There is little discussion - primarily there are unanswered cries for help. Indeed, more useful information about JBoss has been posted on TSS then has ever been posted on your own forums. The detailed semantics of much of your code has been posted _here_ - far more detailed than what is available anywhere on jboss.org. Best of all, all the information is available here for free without paying for thousands of dollars for training :-)

    Indeed, the most common posts on your forums that have more than 0 replies are the ones where someone cries for help, and someone else chimes in (often months later) "Did you ever figure this out. We are having the same problem.".

    The very few "discussions" often include beauties like this:

      "You are asking the question. the wrong way.

      Show me what you know what you are doing,
      and prove you have debugged it yourself. I don't hold hands."

    An added bonus of going non-jboss.org is that you don't get insulted on a regular basis - alot of your own employees are quite fond of being exceptionally rude on your forums. Their agenda appears to be more to prove how smart they are, and how pathetic people's questions are, with almost no interest in actually helping people solve their problems.

    Oh yeah, your forums are a barrel of laughs for someone with a problem, Marc!

    If you want to look at useful discussion forums, you might want to investigate BEA's usenet groups, various forums, and the mountain of docs and forums available from IBM - in addition to information on TSS. In some cases the crocodiles, as you call them, have been far more helpful to people in using JBoss software than your own employees are.

        -Mike
  99. Core competency[ Go to top ]

    It's sad that all we get from this JBoss 4.0 roadmap on TSS is non-constructive babble Why not ask some constructive questions like "HOw does MicroJBoss compare to other lightweight containers and IoC?" "What will it take to implement JSR-88?" "What exactly has been implemented for web services so far?"

    >
    > Bill

    Man who cares about how far you are with 1.4 stuff?? You will never hit compliancy in this century. That's much too much load for you little tiny gangsta company.

    Seriously all what we would like to see is WHEN WILL YOU FINALLY PISS OFF ??? Hopefully with a LOAD OF DEBT !!!

    PISS OFF !!!
  100. Core competency[ Go to top ]

    \Bill Burke\
    Our core competency is middleware. I've lived and worked through the birth,life, and death of DCE and CORBA (at DCE and CORBA companies) and am seeing the signs that J2EE is entering its twilight as well.
    \Bill Burke\

    I'm not sure that I'd say that CORBA's dead, but I get your meaning. I'd go further and say it was never all that successful, and that it just peaked pretty low to start with, and is just naturally sliding downwards...

    \Bill Burke\
    When you start seeing specs become bloated and tailored to individual vendors, or take years (J2EE 1.4) to come out with a final release, see open source and proprietary products come out to fill out the numerable holes in the spec, then you know the death is near. The only thing slowing the death of J2EE is a bad economy.
    \Bill Burke\

    I disagree. Too many people say "J2EE" when they _really_ mean "EJB" - and I strongly suspect that this is precisely what you're doing. Cut EJB out of the equation, and frankly J2EE is going very strong. Cut out EJB, and by any metric J2EE is continuuing to gain momentum in the industry.

    \Bill Burke\
    Our microkernel is a strong implementation to rival even the other containers you talk about. We handle component depencies, life cycle, configuration, classloading, hot deployment, and have robust connector architecture to provide remoteness, security and transactions, and finally have an abstraction implementation so that developers can write plugins to deploy their own types of components and get automatically the features I described above. What we haven't done is package and productize our microkernel. There's also a tiny bit of work to do to make it more POJO friendly as you've seen in the 4.0 roadmap.
    \Bill Burke\

    It's a bit of a stretch to call what you have pre-4.0 a "microkernel". There are too many dependencies to merit that moniker.

    Let's look at the rest: dependencies, life-cycle, config, classload, hot deployment, connectors. Pull out "connectors" for a moment, and _everyone_ does the same thing. Even hot-deploy becomes somewhat of a non-issue if you have a true light-weight container - it can be faster and easier to cycle the server than to bother with hot deploy.

    Connector stuff is interesting, and may be a clear advantage of JBoss over some other non-EJB solutions when you talk of integration.

    But seriously - strip out EJBs, and JBoss has a reasonable implementation, but so do many other players. Strip out EJBs, and you've widen the number of products that can directly compete with you. You can argue plusses and minuses, of course, but now you're on a level playing ground.

    \Bill Burke\
    We see Aspect-Oriented Middleware as the future. POJO based development is the future, gives you the highest flexiblity, and allows you to isolate business logic from middleware.
    \Bill Burke\

    If it's so important, how come there's been no development in that area at all except for some modest work in TreeCache for the past 4 months? It's a nice speech Bill, but CVS commits speak louder than words.

    \Bill Burke\
    To us, AOM, is mostly a repackaging of most of our existing technology. AOP is intuitive to us as we've been using aspect oriented techiques when we didn't know it was called AOP.
    \Bill Burke\

    How very, very convenient. We were doing AOP all along and didn't even know it!! Give it a rest, injecting interceptors into a J2EE lifecycle isn't AOP. It's just interceptors.

    \Bill Burke\
    2EE a la carte via AOP, is taken directly and easily from our existing code base. AOP Remoting and AOP clustering, built apon the services already present in our 3.2 and 4.0 codebases, caching(Bela's stuff), although new, we are applying it in a pluggable fashion to our J2EE container in the CMP and SFSB arenas. We're also converting some existing interceptors in our J2EE implementation to aspects(i.e. transactional synchronization/locking).
    \Bill Burke\

    So far, you've actually coded very little of the above. And your roadmap shows J2EE 1.4 taking precedence. You personally haven't touched jboss-aspects or jboss-aop since DR2 back in June/July. All the objective evidence directly contradicts the marketing spin you're issueing here.

    \Bill Burke\
    Finally, Hibernate completes our POJO strategy by providing a robust, flexible, and powerful persistence mechanism. These guys are so good, they were invited to the JDO 2.0 expert committee.
    \Bill Burke\

    This is very interesting work to be sure. But as you say, Gavin's on the JDO 2.0 expert committee, and he's listed as 5% complete on the CMP via Hibernate stuff on the roadmap. From where I sit it looks like it's going to be a long time before we see the benefits you're talking about beyond vanilla Hibernate today.

    \Bill Burke\
    It's sad that all we get from this JBoss 4.0 roadmap on TSS is non-constructive babble Why not ask some constructive questions like "HOw does MicroJBoss compare to other lightweight containers and IoC?" "What will it take to implement JSR-88?" "What exactly has been implemented for web services so far?"
    \Bill Burke\

    I don't think the "babble" is non-constructive. Posting the roadmap was a good thing, and the resulting conversations around it were really inevitable. Get used to it - when projects or companies commit details in writing, some people are going to actually read it and challenge bits that seem fluky to them. You don't think everyone is just going to blindly accept it, do you?

    On that latter - why would people ask those questions? Frankly, you've conditioned me and some others to expect marketing drivel when those questions get asked, and hostility when you're challenged on some of the technical details. Why ask the question when it's been proven beyond a doubt that the resonse is going to be shallow and followups dismissive? If you wish to prove me wrong, then assume the question as asked, and post such things on the Jboss site and/or your Blog as white papers or design notes. You might want to take a page from Bela's book and distribute more documentation on underlying designs and goals of your work in detail. If you want people to truly understand where you're coming from, and you want to get the information out to the public, you have to communicate it. Professionals don't just write code and keep ideas in their head, professionals also cultivate their written communications skills and learn how to write engaging and persuasive documents.

         -Mike
  101. JBoss claims that J2EE is dead[ Go to top ]

    Bill:
    Our core competency is middleware. I've lived and worked through the birth,life, and death of DCE and CORBA (at DCE and CORBA companies) and am seeing the signs that J2EE is entering its twilight as well. When you start seeing specs become bloated and tailored to individual vendors, or take years (J2EE 1.4) to come out with a final release, see open source and proprietary products come out to fill out the numerable holes in the spec, then you know the death is near. The only thing slowing the death of J2EE is a bad economy.

    JBoss now claiming (and almost wishing) the imminent death of J2EE... now that's a good one. Especially since you are still actively pursuing the license.

    You'd better hope none of your customers read this forum.

    Nevertheless, you are right about DCE and CORBA but you omit one very critical thing when comparing these technologies with J2EE: neither DCE nor CORBA ever came close to achieving J2EE's market penetration.

    Anyway. I am still under the shock of your above statement, as should everybody reading this.

    These guys are so good, they were invited to the JDO 2.0 expert committee.


    As far as I can tell, this committee doesn't exist yet and when it does, it's unlikely any of these individuals will be part of it since they need to be members of the JCP. Exceptions for individuals are extremely rare (Richard and Doug made it just recently) but at the end of the day, it will be up to Sun to make this decision, not you.

    --
    Cedric
  102. JBoss claims that J2EE is dead[ Go to top ]

    Bill:

    > Our core competency is middleware. I've lived and worked through the birth,life, and death of DCE and CORBA (at DCE and CORBA companies) and am seeing the signs that J2EE is entering its twilight as well. When you start seeing specs become bloated and tailored to individual vendors, or take years (J2EE 1.4) to come out with a final release, see open source and proprietary products come out to fill out the numerable holes in the spec, then you know the death is near. The only thing slowing the death of J2EE is a bad economy.
    >

    > JBoss now claiming (and almost wishing) the imminent death of J2EE... now that's a good one. Especially since you are still actively pursuing the license.
    >

    ROTFL! You're just scared that we're killing you in the market. Don't worry, Marc has continually made you offers over the years, so you'll have some job security once the layoffs start. ;-)

    BTW, please have a sense of humor here as I am just ragging.....

    > You'd better hope none of your customers read this forum.
    >

    Most of our customers are well aware of our long term vision and it excites them that we have a direction, and a vision. I've said long before that code generation tools and sophisticated IDEs are not the answer to alleviate the complexity of J2EE or middleware in general. A complete architectural overhall is needed instead. Our opinion is that it is AOP and POJO based programming that will lead this. There is just so much promise for AOP to compartmentalize application development even further (system vs. business logic), increase code flexibility, and speed iterative development. The high interest in POJO based frameworks like PicoContainer, Spring, Hibernate, etc... just validates our vision IMNSHO. Even BEA's announced support for AspectJ or hiring of AspectWerk's Jonas Boner just validates again what we're doing IMNSHO.


    > Nevertheless, you are right about DCE and CORBA but you omit one very critical thing when comparing these technologies with J2EE: neither DCE nor CORBA ever came close to achieving J2EE's market penetration.
    >

    Although you have a great point about market penetration, its all part of the natural evolution of things, IMHO.

    > Anyway. I am still under the shock of your above statement, as should everybody reading this.
    >

    Have you been living under a rock?

    > These guys are so good, they were invited to the JDO 2.0 expert committee.
    >

    >
    > As far as I can tell, this committee doesn't exist yet and when it does, it's unlikely any of these individuals will be part of it since they need to be members of the JCP.

    AFAIK, they were invited. BTW, JBoss Group is a JCP member and Gavin is our employee.

    Bill
  103. JBoss claims that J2EE is dead[ Go to top ]

    Bill:
    ROTFL! You're just scared that we're killing you in the market.

    I think the first time I heard Marc say this was three years ago.

    Anyway, you are right, we are scared of the competition. As all sane players in a competitive market such as J2EE should be. Think about it.

    --
    Cedric
  104. JBoss claims that J2EE is dead[ Go to top ]

    Bill:

    > ROTFL! You're just scared that we're killing you in the market.
    >

    > I think the first time I heard Marc say this was three years ago.
    >
    > Anyway, you are right, we are scared of the competition. As all sane players in a competitive market such as J2EE should be. Think about it.
    >
    > --
    > Cedric

    Yes, competition is fierce, fun and exciting. Don't you agree? It's good to be alive and fighting!

    Bill
  105. JBoss claims that J2EE is dead[ Go to top ]

    Cedric said:> Anyway. I am still under the shock of your above statement, as should everybody reading this.
    >

    Bill Burke response: "Have you been living under a rock?"

    If Cedric is living under a rock than I am living under the same one. Outside of some of the comments on this site, and of course the Microsoft rhetoric, I don't see signs of J2EE dying. In fact, it is just now arriving at the point it needs to be for productive development of business applications.
    It is always easy to get caught up in the vision of the next great programming model, but each vision brings its own set of (new)problems. J2EE went through that cycle but has matured somewhat. Most importantly, there are a number of solid products that have implemented it. While the JBoss innovations are interesting, using them is risky because JBoss just does not have the credibility and stability for a large corporation to rely on. It is fine for J2EE applications because I know I can always port to WebSphere or Weblogic if JBoss falls apart (which I think has a 50/50 probability). As a general rule however, I wouldn't use any of their(your) innovations or extensions unless it provided dramatic gains in productivity over what I can get by going with WebSphere, WebLogic or Oracle. So far, I haven't seen anything that approaches that.
    All of this is not meant to disparage JBoss. You have developed a good product and have some intriquing ideas. Realistically however, few companies can tolerate the many risks in basing critical applications on a JBoss programming model. You guys should focus first on what got you where you are, J2EE.
  106. JBoss claims that J2EE is dead[ Go to top ]

    If Cedric is living under a rock than I am living under the same one. Outside of some of the comments on this site, and of course the Microsoft rhetoric, I don't see signs of J2EE dying. In fact, it is just now arriving at the point it needs to be for productive development of business applications.

    > It is always easy to get caught up in the vision of the next great programming model, but each vision brings its own set of (new)problems. J2EE went through that cycle but has matured somewhat. Most importantly, there are a number of solid products that have implemented it. While the JBoss innovations are interesting, using them is risky because JBoss just does not have the credibility and stability for a large corporation to rely on. It is fine for J2EE applications because I know I can always port to WebSphere or Weblogic if JBoss falls apart (which I think has a 50/50 probability). As a general rule however, I wouldn't use any of their(your) innovations or extensions unless it provided dramatic gains in productivity over what I can get by going with WebSphere, WebLogic or Oracle. So far, I haven't seen anything that approaches that.
    > All of this is not meant to disparage JBoss. You have developed a good product and have some intriquing ideas. Realistically however, few companies can tolerate the many risks in basing critical applications on a JBoss programming model. You guys should focus first on what got you where you are, J2EE.
    >

    John,

    dual line problem again. J2EE is important. Not from a technology standpoint but from a market standpoint. From a techie standpoint there are better ways to skin that cat as bill shows at length. Also i recommend you guys pick up the latest JDJ where Bill has an excellent invited paper. Transparent middleware today is done with AOP as band-aid to patch lack of behavior at the VM level (namely state tracking capabilities and Advisable as an Object method, the first one is already a debugging feature the second one takes more work)

    I think one of the problems of J2EE is that it DEFINES STANDARDS BY CONTRACT (interfaces). I don't want to get into the pros and cons of all of it, but I am growing convinced that design by interfaces lead to bloat. Take EJB again, implementing this and that interface method and knowing about all the callbacks is cumbersome and weak. THERE ARE TRANSPARENT WAYS TO ACHIEVE THE SAME THING.

    On the 50/50 risk of JBoss inc going away. Relax, we were here in 1999, we were here in 2000, we are still here in 2004. In 2004 we are going lightspeed as in starwars. JBoss and JBoss inc are here to stay. I have been at this for the past 5 years and I got 15 to go.

    Keep in mind that the code is free software, as in LGPL, not the BSD crap that can be closed on you in derivative works (anyone here remember how lutris died overnight because the VCs wanted the source closed?). JBoss is true FREE SOFTWARE in the FREE SOFTWARE FOUNDATION sense. The code can't go away. The code won't die as long as people use it/maintain it. Imagine all of JBoss being abducted by aliens, guaranteed there are 1000 consulting firms springing up in our place.

    OUR PRODUCTION USAGE is close to 25% in the USA according to an SDTimes survey.

    Relax, professional open source is a 'deep' movement, not just a flash in the pan where we party like 1999. We are children of the great tech depression of 2001. We have been at it almost 5 years as I said and many more to come.

    I believe middleware in general (J2EE and beyond) is a 20 year movement and we are 5 years into it. Professional open source may live even longer. At least we are trying our darn best and it is working so far. so chill.

    That being said, regardless of the company viability, which again is great, your businesss is appreciated. I see some posts here that say "I moved from BEA to JBoss and it works just fine", great make sure to get your production support so we can keep on delivering the goods and you can have you ass covered.

    I do understand the fear of going with "proprietary technology". To that we answer 1- it is FREE software so the lock in is weak 2- we are trying to standardize a lot of what we do. Already EJB3.0 is going in that direction.

    make your own opinion, but rest assured we are committed and long term,

    marcf
  107. Bill Burke's AOP article[ Go to top ]

    Also i recommend you guys pick up the latest JDJ where Bill has

    > an excellent invited paper. Transparent middleware today is
    > done with AOP as band-aid to patch lack of behavior at the VM level
    > (namely state tracking capabilities and Advisable as an Object method,
    > the first one is already a debugging feature the second one takes
    > more work)

    Bill Burke's AOP article is available online.

    Title: It's the Aspects
    http://www.sys-con.com/story/?storyid=38104&DE=1
  108. JBoss claims that J2EE is dead[ Go to top ]

    Marc Fleury :

    > Keep in mind that the code is free software, as in LGPL, not the BSD
    > crap that can be closed on you in derivative works (anyone here
    > remember how lutris died overnight because the VCs wanted the
    > source closed?). JBoss is true FREE SOFTWARE in the FREE SOFTWARE
    > FOUNDATION sense. The code can't go away. The code won't die
    > as long as people use it/maintain it. Imagine all of JBoss being
    > abducted by aliens, guaranteed there are 1000 consulting firms
    > springing up in our place.

    IIRC, there is no difference in BSD vs LGPL code in terms of what can happen to a future version. Any code that is has been released under LGPL or BSD license is yours to do what you wish with, as long as you follow the requirements of the respective licenses. There is nothing to prevent the copyright owner of any LGPL or BSD-licensed work from closing *future* versions of the code, or as in the case of MySQL, offering the code under a different license in parallel w/ an OSS license for some consideration (like a fee). In neither case can re-licensing be done retroactively, so there is no difference in risk. (Actually, I'm not sure if that's 100% true w/ the LGPL due to section 13 of the LGPL, but IANAL nor really interested in the LGPL)

    geir
  109. JBoss claims that J2EE is dead[ Go to top ]

    Marc Fleury :

    >
    > > Keep in mind that the code is free software, as in LGPL, not the BSD
    > > crap that can be closed on you in derivative works (anyone here
    > > remember how lutris died overnight because the VCs wanted the
    > > source closed?). JBoss is true FREE SOFTWARE in the FREE SOFTWARE
    > > FOUNDATION sense. The code can't go away. The code won't die
    > > as long as people use it/maintain it. Imagine all of JBoss being
    > > abducted by aliens, guaranteed there are 1000 consulting firms
    > > springing up in our place.
    >
    > IIRC, there is no difference in BSD vs LGPL code in terms of what can happen to a future version. Any code that is has been released under LGPL or BSD license is yours to do what you wish with, as long as you follow the requirements of the respective licenses. There is nothing to prevent the copyright owner of any LGPL or BSD-licensed work from closing *future* versions of the code, or as in the case of MySQL, offering the code under a different license in parallel w/ an OSS license for some consideration (like a fee).
    >

    2 different things copyright and license.

    the mySQL thing works because mySQL owns 100% of the code (full copy-rights as it is theirs). One of the rights of the owner of the copy is to license the code under whatever license they want. mySQL licenses to the world at large under GPL and then licenses for those who don't want the GPL under different terms (for pay proprietary). It is not a license issue it is a feature available to mysql because of 100% copyright ownership.

    HOWEVER, the JBoss case

    IF YOU OWN 99% (JBoss inc made 95% of the commits recently), then 1% comes from someone that OWNS that 1% and licenses it to you under the LGPL (for example). Under the LGPL I can't take that 1% create a derivative work of the whole and license it under a proprietary license. I just can't, i must license under the LGPL. That presence is a check, for ever. If it is a BSD license then you can take that 1% and create a closed version. Domain of application? most of jakarta. IBM is adamant on BSD licenses there so they can create closed versions of tomcat that get rebranded 'websphere'.

    Another example is Apple, where they created a closed GUI on BSD (macOSX) but don't give anything back, it helps Apple, but the user needs to pay for that derivative and the original developer doesn't get jack, not even source code back. Let's not get into the argument that GUIs are expensive to build, the failure of linux to create anything resembling macOSX GUI proves it. If tomcat was LGPL or GPL under JBoss.org (for example :) then websphere would need to be LGPL itself. Is it clear?

    If the license is LGPL, it says you must give back to the community and hence it doesn't allow for a complete closing of the source, BSD allows for a closing without giving back.

    Bottom line?

    BSD is VENDOR FRIENDLY
    FSF is DEVELOPER/USER FRIENDLY.

    marcf

    if you own 99% of the code

    If one owns 100% of the code (copyright wise) then he can license under whatever he wants, think mySQL selling PROPRIETARY LICENSES.

    If you don't then you can't take LGPL code from someone else and close a derivative.
  110. JBoss claims that J2EE is dead[ Go to top ]

    Marc Fleury :
    > HOWEVER, the JBoss case
    >
    > IF YOU OWN 99% (JBoss inc made 95% of the commits recently), then
    > 1% comes from someone that OWNS that 1% and licenses it
    > to you under the LGPL (for example).

    I'm not quite sure what you mean here.

    JBoss Group LLC doesn't own 99% of the JBoss code just because JBoss Group LLC employees made 95% of the recent commits, unless it was brand new code.

    Otherwise, that work has the combined ownership of all the authors of the code. You can't take away ownership that came before by doing a commit.

    > Under the LGPL I can't take that 1% create a derivative work
    > of the whole and license it under a proprietary license.
    > I just can't, i must license under the LGPL. That presence
    > is a check, for ever.

    Unless you get the authors of that code to grant you permission to do that. Then you can.

    > If it is a BSD license then you can take that 1%
    > and create a closed version. Domain of
    > application? most of jakarta. IBM is adamant
    > on BSD licenses there so they can create
    > closed versions of tomcat that get rebranded 'websphere'.
    >

    And I'm not sure anyone gives a hoot. If I'm a user of Tomcat, my rights don't get taken away when IBM does this. I can *choose* to switch to IBM's servlet implementation (it's not Tomcat any longer, btw) or not. My choice as the user/customer.
     
    >
    > If the license is LGPL, it says you must give
    > back to the community and hence it doesn't
    > allow for a complete closing of the source,
    > BSD allows for a closing without giving back.
    >

    Be careful Marc. You won't want JBoss customers to think that there's some kind of license leakthrough on the JBoss API's, putting their corporate systems and proprietary technology that's based on JBoss under the LGPL.

    Yes, BSD-derived licenses give companies the freedom to close without giving back. Everyone writing BSD-licensed code knows this, and has made the decision to contribute their work this way, and it's not a big deal, because for the users, there's no additional risk, but plenty of benefit.

    You characterize the situation where the next version of something BSD-licensed becoming closed as risky for users, but because they have what I assume is an acceptable version to work with, including all code, they can chose to stay where they are. They aren't forced to move.

    This is no different than if JBoss Group LLC decided to market and support closed-source components/enhancements/additions to the LGPL-ed code base, which they could do at any time. Your customers would choose your new closed-source stuff or not. You can't force them.
     
    > Bottom line?
    >
    > BSD is VENDOR FRIENDLY
    > FSF is DEVELOPER/USER FRIENDLY.
    >

    If you believe this, why not do all subsequent work under the GPL?

    geir
  111. JBoss claims that J2EE is dead[ Go to top ]

    Geir, JR,

    > JBoss Group LLC doesn't own 99% of the JBoss code just because JBoss Group LLC employees made 95% of the recent commits, unless it was brand new code.
    >
    > Otherwise, that work has the combined ownership of all the authors of the code. You can't take away ownership that came before by doing a commit.
    >

    ?

    I don't claim it, I am fact saying exactly that. Ergo we can't change the license so the WHOLE remains LGPL.


    >
     And I'm not sure anyone gives a hoot. If I'm a user of Tomcat, my rights don't get taken away when IBM does this. I can *choose* to switch to IBM's servlet implementation (it's not Tomcat any longer, btw) or not. My choice as the user/customer.
    >

    I give a hoot. Most our contributors give a hoot. All linux developers give a hoot. MySQL gives a big hoot.

    L/GPL >> BSD from a developer standpoint. Don't get me started on the "it is more freeeeee" gay argument. I develop code that I KNOW cannot be taken by IBM or BEA outright extended and closed (granted BEA still copies the ideas). On the SCO-IBM fiasco I am on the side of the guy that has the IP. IP is property and property is important. Free software creates a community of IP, it doesn't destroy it like microsoft claims in fact it is VERY STRONG on IP. BSD is weak on IP.

    It means that if a vendor makes an enhancement on what the community does it must give back to the community. That is simple. And good.  It beats contributing in a vacuum for IBM to take it away and say "thank you, want a tshirt that says (I am with stupid) ?". That being said IBM does so many things (Eclipse) that I guess it is unfair to call them out on that. But here we are looking out for our interest as a body of developers more that we are looking out for a 3rd party vendor while singing hare krishna. BSD is a like a sect.

    On the USER side. It sure doesn't prevent you from using Tomcat, but if the kick ass clustering HTTPSession wasn't done by JBoss under LGPL for FREE you would have to shell out big dolares for the first 2 bit implementation of proprietary software crap by Mr SUN. How is that good for YOU Einstein? huh?

    > Be careful Marc. You won't want JBoss customers to think that there's some kind of license leakthrough on the JBoss API's, putting their corporate systems and proprietary technology that's based on JBoss under the LGPL.
    >

    ? LGPL is clearcut, use/embed/extend/link the binary all you want and you are safe. Modify the source and you must LGPL your work. Linux, mysql, JBoss all FSF stuff.
     
    > Yes, BSD-derived licenses give companies the freedom to close without giving back. Everyone writing BSD-licensed code knows this, and has made the decision to contribute their work this way, and it's not a big deal, because for the users, there's no additional risk, but plenty of benefit.
    >

    There is, see above. For the developer there is no payback in code. For the user there is no guarantee that the kick ass version of it will be free as well. It is as simple as that. At least in shared ownership the FSF licenses make sure no one owner can close (which is not the case of mySQL as you point out).

    Also don't you find it interesting that all emperical evidence says that SUCCESSFUL BUSINESS IN OPEN SOURCE IS ALL ON FREE SOFTWARE (Linux RH/Suse, mySQL, JBoss etc). Free software leads to business who would have thought?

    > You characterize the situation where the next version of something BSD-licensed becoming closed as risky for users, but because they have what I assume is an acceptable version to work with, including all code, they can chose to stay where they are. They aren't forced to move.
    >

    True but in your case you have to pay to move. I am not saying that paying is bad, heck I run a business here, but HOW IS THAT BETTER FOR THE USER THAN GETTING IT FOR FREE?
     
    > This is no different than if JBoss Group LLC decided to market and support closed-source components/enhancements/additions to the LGPL-ed code base, which they could do at any time. Your customers would choose your new closed-source stuff or not. You can't force them.
    >

    You are correct, we could create closed 'addons' at any time, in fact many of our partner do, in monitoring, perf analysis, web services bla bla bla ... but we don't.

    Maybe it is worth I spend a couple of seconds on explaining why it takes us 23 seconds to gun down proposals for proprietary GUIs (we get one a quarter on average). The reason is that we can grow on maintenance revenue, grow grow grow the revenue in maintenance is good.

    Think BEA 100% license + 20 % recurrent maintenance. Take away the license, keep the 20% recurrent and you have our business... see? no need to be greedy.
      
    Remember the goose that laid the golden egg.

    > > BSD is VENDOR FRIENDLY
    > > FSF is DEVELOPER/USER FRIENDLY.
    > >
    >
    > If you believe this, why not do all subsequent work under the GPL?

    He, good point I bang my head against the wall every morning on that one. And then I relax when I think of ISVs. The CEO of mySQL makes fun of me for being a LGPL weenie. We could in fact have a succesful business on the GPL dual licensing scheme ALONE, like mySQL does. There are historical reasons that put us in the LGPL camp and there we are.

    But the silver lining is a great penetration of the OEM market. An analyst called us up and said "you are #1 in OEM, how come?". The answer in my mind is 1- modular 2-free 3- LGPL. The LGPL ALLOWS EMBEDDING AS LIBRARY and the OEM benefit from the FSF feedback license (meaning they won't be stuck with free crap and for-pay derivative). NO BAIT AND SWITCH. So we are doing the JASP program to help them sell support in their channel.

    Let me give you an example. The other day McDonald's (yeah the burger company) calls us up, they use this EDI package from a 3rd party vendor. The package powers 300 of the fortune 500 companies (you figure it out :). Turns out the product is based on JBoss. They need the support and we are the only ones and they want to buy. What does that tell you? PENETRATION == BUSINESS. Open Source has put penetration within reach of the small guy.
      
    LGPL is the perfect license for the ISV market.

    marcf
  112. LGPL vs BSD FUD[ Go to top ]

    Marc Fleury :

    > IF YOU OWN 99% (JBoss inc made 95% of the commits recently),
    > then 1% comes from someone that OWNS that 1% and licenses it to you
    > under the LGPL (for example). Under the LGPL I can't take that 1%
    > create a derivative work of the whole and license it under a
    > proprietary license. I just can't, i must license under the LGPL.

    Not true (at least in my non-legal opinion).

    The LGPL depends on the definition of a derivative work. You can contribute
    code to a project without that code being a derivative work of the project.
    As such, the copyright holder is free to reuse and relicense their code
    anyway they like.

    Definition of a derivative work is tricky - see the ongoing debate about
    linux binary modules to get a flavour of this - but just being part of
    the same CVS repository is not sufficient to define a derivative work.
    The LPGL says:

      "mere aggregation of another work not based on the Library with the
      Library (or with a work based on the Library) on a volume of a storage
      or distribution medium does not bring the other work under the scope
      of this License."

    So for example, the fork of Jetty the JBoss 3.2.1 CVS code base (9.1% of
    the code) is not a derived work of any LGPL code and remains under
    it's Artistic license. However, the Jetty/JBoss integration code is
    definitely a derived work from JBoss AbstractWebContainer and thus
    needs to respect the LGPL license of that class.

    Just being part of JBoss does not mean a copyright holder gives away
    any rights to any original work. Moreover, even if a copyright holder
    does breach the LGPL, that does not restrict their rights to their own
    work - it only invalidates their license to use and distribute the
    original work. Nothing in the GPL or LGPL requires a copyright owner
    to give away ownership of their code in anyway.



    > IF YOU OWN 99% (JBoss inc made 95% of the commits recently)

    I have to do a double take on this line...

    Is this meant to be a good thing? You need to encourage an
    open community if you want to keep JBoss a quality project.

    Also the juxtaposition of the two statements is also a little
    suspect considering recent claims and discussions. Issues of
    code ownership and current commit access are separate and control
    of CVS does not give one control of the code.
  113. Short memory[ Go to top ]

    Marc:
    On the 50/50 risk of JBoss inc going away. Relax, we were here in 1999, we were here in 2000, we are still here in 2004.

    Ah come on Marc, did you already forget Telkel?

    Not that it matters anyway, if JBoss Group fails, you will probably create a third company. But please, no revisionism.

    --
    Cedric
    http://beust.com/weblog
  114. Short memory[ Go to top ]

    Ah come on Marc, did you already forget Telkel?


    What's Telkel?
  115. Short memory[ Go to top ]

    Marc:

    > On the 50/50 risk of JBoss inc going away. Relax, we were here in 1999, we were here in 2000, we are still here in 2004.
    >

    > Ah come on Marc, did you already forget Telkel?
    >
    > Not that it matters anyway, if JBoss Group fails, you will probably create a third company. But please, no revisionism.
    >

    you zoolander, I am not "revisioning" or "eugogolizing" </zoolander reference>.

    Telkel was my first company, the one that funded the development of early JBoss (me full time) to basically do a hosted jboss. Think marc andressen's loudcloud only with JBoss instead of BEA. It didn't work.

    Our first employee was a young Rickard Oberg. Since then he has grown a scruffy beard and personality, oh well, I prefer bill for friendly scruffiness. If you want more details read blue and white.

    The good that came out of Telkel was a JBoss 2.0 prototype courtesy or Rickard and me. It took Scott Stark my first partner in JBoss Group to make JBoss 2.4 (aka a real product) and with it business FROM THE FIELD came dribbling in. We were coackroaches of business, surviving the cold nuclear winter of 2001. Let the heroic trumpet music raise in the back. Good new is I didn't have to invent a business plan this time around. The customer gave me the business plan, see business week article for all the gory details.

    BUT THE REAL POINT IS

    When I say WE I mean JBoss. I mean JBoss the project and product. And the above statement is true. Not in a bill clinton "true by my standard" kind of way, really really true. JBoss has been there for all that time and growing. And today JBoss == JBoss inc. You figure it out.

    marcf
  116. Short memory[ Go to top ]

    Indeed, very short....

    \MF\
    JBoss == JBoss Inc. You figure it out.
    \MF\

    How does one explain Elba, then? It is the quintessential "JBoss by another name". In this case "Elba != JBoss Inc", and yet "Elba source code == JBoss source code". Bits of JBoss even live on in Geronimo, thanks to the fact that the Geronimo guys were the actual copyright holders of the source files in question within JBoss.

    You still don't get it Marc, but you've fundamentally based your business on IP which you do not, and never have, owned. You give some kind of unique training on JBoss - even though the majority of the actual authors don't work for JBoss, not to mention the fact that any fool can just download all your code and figure out how it works for themselves.

    This model works for something like RedHat, because Linux is gigantic (truly gigantic - if linux is a baseball, JBoss is one atom inside of it). Plus it's complex - beyond all the various interdependencies and drivers and models one can try out, an operating system and layered on tools written in C and C++ is not something you figure out quickly.

    None of this applies to JBoss. You've got plain old Java code doing fairly routine things. People who have no hope in ever understanding how Linux ticks under the covers can (and have!) quite easily read the JBoss source code, understood it, and even changed it. You crow how big Jboss has gotten, but in the scheme of things it's a rather small code base really - doubly so when all web stuff doesn't come from JBoss at all, but instead Jetty or Tomcat.

        -Mike
  117. JBoss claims that J2EE is dead[ Go to top ]

    Marc>> J2EE is important. Not from a technology standpoint but from a market standpoint.

    I am more interested from the Business perspective. That is where the risk must be justified
     
    Marc>> On the 50/50 risk of JBoss inc going away. Relax, we were here in 1999, we were here in 2000, we are still here in 2004.

    Your confidence is great, as it must be to compete, but that doesn't change anything in the end. There have already been far better J2EE servers than JBoss that have nearly disappeared for various reasons. Technology might have been great but a business can't rely on an unstable company no matter how good the technology is.

    Marc>> Keep in mind that the code is free software, as in LGPL, not the BSD crap that can be closed on you in derivative works (anyone here remember how lutris died overnight because the VCs wanted the source closed?). JBoss is true FREE SOFTWARE in the FREE SOFTWARE FOUNDATION sense. The code can't go away.

    Free code is nice, but I want a product that will be supported into the future. Once support goes away, that free code becomes the expensive to maintain (if it is even possible). Here is where I rely on J2EE compliance to ensure a manageable migration to some other server.

    Marc>>The code won't die as long as people use it/maintain it. Imagine all of JBoss being abducted by aliens, guaranteed there are 1000 consulting firms springing up in our place.

    That is the risk I am talking about. JBoss popularity could decrease very rapidly, and I would be left with a pile of unsupported code. I don't worry about that at all with IBM, and very little with BEA.

    Marc> it is FREE software so the lock in is weak

    I don't follow that logic. A lock-in is a lock-in. In some ways it is worse when you are locked into "free" code because you end up supporting it yourself. I write business applications and have no interest in becoming a middleware support organization.

    One last comment non-technical comment. I noticed your use of the term "gay" in several posts on this thread. That is immature as well as offensive.
  118. JBoss claims that J2EE is dead[ Go to top ]

    Marc> it is FREE software so the lock in is weak

    >
    > I don't follow that logic. A lock-in is a lock-in. In some ways it is worse when you are locked into "free" code because you end up supporting it yourself. I write business applications and have no interest in becoming a middleware support organization.

    I think Marc is making a mistake when he says that the lock-in is weak because the software is FREE. The software is not really free right now, it's just released under a free license. Let me say that again: this is one case where the spirit of a free license has been violated and the software in question is not truly free.

    Think about how this usually works: the free license ensures that if you contribute code it will never be locked up. This really is what makes people contribute. Marc is right that there is a big difference between BSD and GPL. Just look at the difference between BSD Unix and Linux. The only reason why Linux took off is because people know that they are not going to get ripped off.

    The license is important, but in the end it only works because it makes people do the right thing: it creates the perception of a product that will be around forever. And this is not the case with JBoss. As we have read, Marc says that JBoss = JBossGroup.

    Judging from the fact that Jonas is still around, that Geronimo is starting up, and the fact that Sun is shipping their app server for free, I'd say the future doesn't look too good for JBoss. It could go under, in the sense that people will simply stop using it. If the code really is in bad shape, and marketing issues prevent anyone from fixing it (and fall behind against BEA/IBM), then it could just become another dead code base nobody wants.
  119. JBoss claims that J2EE is dead[ Go to top ]

    John,

    >
    Technology might have been great but a business can't rely on an unstable company no matter how good the technology is.
    >

    Technology greatness is the basis for our business. That and the people. All JBoss project leaders are at JBoss inc. HP tried the free product play, it didn't work. FREE is not not enough these days to get any market penetration. The net gives you that opportunity but if you ask me, LUCK is the most important factor :)
     
    >
    Free code is nice, but I want a product that will be supported into the future. Once support goes away, that free code becomes the expensive to maintain (if it is even possible). Here is where I rely on J2EE compliance to ensure a manageable migration to some other server.
    >

    1- We agree on the portability in J2EE that is the point of J2EE risk minimization and opportunity maximization. See previous post on this thread re: WE PORTED FROM BEA TO JBOSS AND IT ROCKS. <sales plug> Our worldwide JASP partners offer porting services (</sales plug>).

    2- We disagree on the risk of availability of SUPPORT. We are building a network of PARTNERS, the JASP partners. JASP stands for JBoss Authorized Services Partners (JASP). These are AUTHORIZED and OFFICIAL and SUPPORTED resellers of our support, people trained in the fine arts of the bushido of JBoss. They do first line and 2nd line and JBoss inc does 3rd line for them, meaning they are certified. We got partners pretty much everywhere in EUROPE (France, Switzerland, Ireland, UK, Holland, Belgium, Italy, Germany, norway, finland), we cover the US as well (US and Canada), we cover APAC (Australia, HongKong, China, Japan), and we are still looking for partners in other regions of the world. Please refer to our website for missing and available countries . In fact one of the reasons we launched the JASP program was to STRUCTURE the worldwide support organization.
    You can see a list of read more about the JASP program.

    3- We agree on the fact that if all the JBoss inc employees, who effectively control JBoss then the JBoss support business is missing a critical piece of it. Remember that JBoss is not kiddie open source, it is professional open source. WE TIGHTLY CONTROL OUR CVS COMMITS AND WHO DOES WHAT. Cameron Purdy got that point right about open source, that it is scary in general cause you need tight control of the codebase to trust it. JBoss is tight, it is open source you can trust (tm). 1st line can be done by many people. However 3rd line knowledge of JBoss effectively resides at JBoss inc.

    4- Did you know that JBoss Certified Programmer, our certified brand was ranked SECOND in fastest growing certification by CRN (Computer Reseller News) AT LARGE SYSTEM INTEGRATORS. That's right, JBoss placed second at large SI for fastest growing . That ranking is very followed in the industry and is yearly, we came 2nd after symantec and before Microsoft DBA certification. That was all the more surprising to us since we WERE NOT MARKETING the certified programmer brand AT ALL. We were launching the JASP program. The certification is part of the JASP. But it indicates MASSIVE GROWTH OF JBOSS DEMAND BY THE FIELD AND JBOSS DEVELOPMENT EXPERTISE AT LARGE SI.

    5- While you are right, knowing how to develop on JBoss and how to do PRODUCTION SUPPORT on JBoss are two different things. The first, development is widely available, the second, intimate knowledge of JBoss is best done by JBoss inc. So let me tell a little story though. About 2 years ago, General Motors did their point of sales implementation and retained CAP Gemini to do it. Cap did the development on JBoss and beta tested on JBoss. It passed. So the Detroit brass asked "tell me Mr cap why should we be paying millions for this BEA thingy you recommend, can you support JBoss?", the reason Cap wanted them to buy is that SI get a 10% kickback from BEA usually (ah, the beauties of business :). But they said "sure we will support it" and of course didn't contact us 2 years ago. We heard the story because a developer left them and came to a training. Can CAP support JBoss? sure 1st line they can, and better than us given their size. Can they do deep 3rd line JBoss? hmmmm I doubt it and that is why we have JASP. Another story is the IRAQ's WAR real time information collection system. Granted it isn't the best reference for europeans, but damn it we proud of the fact that JBoss powered the real time stuff. It was a JBoss grid installation done by Northrup Grunman.

    >
     That is the risk I am talking about. JBoss popularity could decrease very rapidly, and I would be left with a pile of unsupported code. I don't worry about that at all with IBM, and very little with BEA.
    >

    0- Don't worry about our popularity, this thread is at what 140 posts? most announcements get 5 posts? JBoss is visible and you guys make it visible (btw thanks to all of you that help make JBoss real in your companies). JBoss inc is too small and we know it is you guys that make it real, so again thanks)

    1- You are right not to worry about IBM.

    2- I would worry about BEA and everyone else though.

    3- JBoss inc recruits from a large pool of talent that self trains in JBoss and has to prove worth (through commits) before we hire them. We added 2 people on JAN 1st to JBoss inc, both from JBoss. I added Tom elrod in ATL who wrote the JMX remoting framework and Thomas Diessler in germany who wrote the J2EE 1.4 web services stuff. These guys become professional open sourcers after rising through the ranks. BEA is closed, probably not refactored like open source code (see cedric posts about a year ago deriding our constant refactoring of code in open source and him saying what customers need are solutions and all that bla bla bla) but it does lead to modular/clean code and people get in everyday. BEA loses employees every day, just like any company does. But do they have it easier to train talent on the core container? I seriously doubt it. We do, JBoss inc can recruit pre-trained top talent WORLDWIDE. You will see some more coming on board. Professional Open Source is a beacon of sanity for many open sourcers out there. We proved open source can work as a profession and those who get it want in. That validation by fellow open sourcers is something everyone at JBoss inc is very proud of.

    3- You are correct JBoss inc is the best one to support JBoss. But already our partners would spring up and many consultancies. The likelyness of JBoss code being unsupported takes us disappearing (the people) and all our partners. The likeliness of that is not 50/50. You kidding me? JBoss is open source it will be supported as long as someone pays for that support.

    4- OPEN SOURCE IS THE ONLY KNOWN STRUCTURE TO WITHSTAND FULL FRONTAL ASSAULT ON A MARKET BY MICROSOFT. No other company on earth has the money, clout and sheer will to survive. Only OPEN SOURCE has historically survived and thrived in a market assault by MS (linux and apache). So we are all pissing on .NET but I frankly think C# is a kick ass language. It has some features that make it great for AOP for example (the tag stuff is fantastic). The language is great but what is missing are the libraries (JDK is very complete and powerful) and quality of the services (no O/R to speak of... you kidding me?). HOWEVER .NET WILL BE THERE ONE DAY and then what? huh? what? you want to rely on mighty BEA? God help you. I went to IBM to meet the brass there, they say "we see 3 players in the future of middleware, microsoft, IBM, and open source". Be careful what basket you put eggs in. The safe choice may become JBoss very soon.

    5- everyone else has dissapeared through MASSIVE consolidation that is still going on. We haven't, in fact we are one of the reasons for that consolidation. The safe choice may become JBoss very soon.

    6- We will make some announcements re: results and finances of JBoss inc. You will be pleasantly surprised about our financial viability (if you are rooting for us that is :)

    The safe choice may become JBoss very soon :)

    > Marc> it is FREE software so the lock in is weak
    >
    > I don't follow that logic. A lock-in is a lock-in.

    you are right a lock-in is a lock-in but the downside below is false.

    >
    In some ways it is worse when you are locked into "free" code because you end up supporting it yourself.
    >

    That is false, and a myth coming from open source FUD litterature. It is on page 2 of the open source leader manual. Open Source can have more support!!!! see my points above, the pool of JASP partners is widening and the open source nature makes the barriers to entry of support low. The JASP program is our future. The weak comes in there as there is no bait and switch strategy behind it. Most people want monopolistic positions to abuse its power. We don't.

    <evangelical plug>If you are an SI please check our JASP program, where ever you are, we are growing that network and it is still early in the game, join us to make some mullah <evangelical plug>

    >
     I write business applications and have no interest in becoming a middleware support organization.
    >

    totally, that is our business and that of our JASP partners.

    >
    One last comment non-technical comment. I noticed your use of the term "gay" in several posts on this thread. That is immature as well as offensive.
    >

    If you want me to be PC... good luck. I don't mean gay in any offensive way, I apologize if anyone was offended. I don't care about gay though, I couldn't care less. Have worked with many gay men in the past. I FIGURE SKATE for god's sake, our logo sports the rainbow. Do me a favor and don't give PC college lessons here. Peace to the gays.

    regards

    marcf
  120. JBoss claims that J2EE is dead[ Go to top ]

    <evangelical plug>If you are an SI please check our JASP program, where ever you are, we are growing that network and it is still early in the game, join us to make some mullah <evangelical plug>

    >

    you dirty fucking *******. All your "worldwide" fucking JASP partners consist 90% IONA.

    That's what you have: a big dirty greedy stinking mouth.

    PISS OFF !!!
  121. JBoss claims that J2EE is dead[ Go to top ]

    So now it gets a bit more interesting...

    \MF\
    Technology greatness is the basis for our business. That and the people. All JBoss project leaders are at JBoss inc.
    \MF\

    This is an interesting choice of words. A more interesting choice would be "the majority of people who wrote the JBoss code now have nothing to do with JBoss". Please don't cite the number of commits, 75% of which are 1 or 2 line changes. How much of the code in JBoss was written by people who are JBoss Group LLC employees?

    (As an aside, it's a bad idea to refer to your company as JBoss Inc. when it's an LLC)

    \MF\
    1- We agree on the portability in J2EE that is the point of J2EE risk minimization and opportunity maximization.
    \MF\

    I'm not really sure what "opportunity maximization" means in this context, but you're right about J2EE minimizing risk. In fact I think your actual approach, as demonstrated by your roadmap, is the rational one - put J2EE 1.4 in the forefront, and move everything else to "long term". Do just enough playing around with AOP and the like to make you look "innovative", but make sure it doesn't distract from the J2EE 1.4 development work. Smart. Of course that rather shoots people in the foot who are looking at the AOP side, but dems the breaks.

    \MF\
    2- We disagree on the risk of availability of SUPPORT. We are building a network of PARTNERS, the JASP partners. JASP stands for JBoss Authorized Services Partners (JASP). These are AUTHORIZED and OFFICIAL and SUPPORTED resellers of our support, people trained in the fine arts of the bushido of JBoss. They do first line and 2nd line and JBoss inc does 3rd line for them, meaning they are certified.
    \MF\

    Sounds good and sounds reasonable. I've yet to hear from anyone who actually has bought such support, though, good, bad, or indifferent. Anyone _other_ than a JBoss Group LLC employee have any comments on the for-pay support?

    \MF\
    3- We agree on the fact that if all the JBoss inc employees, who effectively control JBoss then the JBoss support business is missing a critical piece of it. Remember that JBoss is not kiddie open source, it is professional open source. WE TIGHTLY CONTROL OUR CVS COMMITS AND WHO DOES WHAT.
    \MF\

    JBoss Group has the JBoss(tm) Trademark, assigned through you, and controls access to _a_ CVS tree. Individuals or corporations are free to replicate your CVS tree and start their own if they like, and there's nothing you could do about that, as Elba has proven. As for "professional open source", that remains to be seen. Putting together the 4.0 roadmap seems to have been an "above and beyond the call of duty" sort of activity, or so it seems from the comments here by JBoss Group LLC employees. Not a normal course of business, but a remarkable extra effort. Likewise, Bela Ban seems to be the only person who actually writes down any of his designs in a coherent fashion. And documentation, as always, is seen as an after thought.

    Most professional product companies have product managers and in-depth development plans, and very impressive design and architectural documents, and mountains of documentation. This is what I expect from a professional effort. Professional means more than banging out code. So far, JBoss' "professional open source" seems to mostly mean that you pay people to bang out code.

    \MF\
    So let me tell a little story though. About 2 years ago, General Motors did their point of sales implementation and retained CAP Gemini to do it. Cap did the development on JBoss and beta tested on JBoss. It passed. So the Detroit brass asked "tell me Mr cap why should we be paying millions for this BEA thingy you recommend, can you support JBoss?", the reason Cap wanted them to buy is that SI get a 10% kickback from BEA usually (ah, the beauties of business :). But they said "sure we will support it" and of course didn't contact us 2 years ago.
    \MF\

    Two years ago, eh? This is the version of JBoss, I take it, that yourself and Mr. Burke say is a piece of crap and needs to be completely rewritten, right? This is a version that contains all that awful, awful code written by now-ex JBossers that Bill Burke derides at every possible moment? I wonder how customers of this implementation faired. Did they save money, or did that 10% off the top (or whatever) from BEA just go into someone else's pocket instead?
    Did anyone really save any money here, or was it a case of money just being diverted from BEA to someone else?

    \MF\
    JBoss inc recruits from a large pool of talent that self trains in JBoss and has to prove worth (through commits) before we hire them. We added 2 people on JAN 1st to JBoss inc, both from JBoss. I added Tom elrod in ATL who wrote the JMX remoting framework and Thomas Diessler in germany who wrote the J2EE 1.4 web services stuff. These guys become professional open sourcers after rising through the ranks. BEA is closed, probably not refactored like open source code (see cedric posts about a year ago deriding our constant refactoring of code in open source and him saying what customers need are solutions and all that bla bla bla) but it does lead to modular/clean code and people get in everyday. BEA loses employees every day, just like any company does. But do they have it easier to train talent on the core container? I seriously doubt it. We do, JBoss inc can recruit pre-trained top talent WORLDWIDE.
    \MF\

    What a load of bull. What an unbelievably, hilariously wrong set of statements. You lost some early people like Rickard, you lost the whole CDN team of people. The life of JBoss has been one of desperately trying to backfill people who have abandoned the project.

    I've talked to both BEA and IBM engineers, and I can tell you from first hand experience they are a very impressive lot. On the IBM side, I've recently had some in-depth discussions with Billy Newport and people from Hursley and other IBMers, and they really, really know their shit. Some of the people involved in Websphere development _literally_ wrote the research papers which lead to some of the leading uses of transaction processing and messaging today. The qualiity of people on the BEA side are almost as impressive. And I can say without hesitation that, as bright as some JBossers are, they are not in the top 1%. WebLogic started small, and Websphere was a victim of the monstrous size of IBM for years, but you are foolhardy if you dismiss them (and especially foolhardy to dismiss IBM). Yes, both companies employ some idiots, I know because I've worked directly with them. But there are some really, really smart folks at the heart of WebLogic and Websphere development - to put it blunty, smarter than anyone JBoss Group LLC fields today.

    \MF\
     The likelyness of JBoss code being unsupported takes us disappearing (the people) and all our partners. The likeliness of that is not 50/50. You kidding me? JBoss is open source it will be supported as long as someone pays for that support.
    \MF\

    This is correct, but is spoken from an odd viewpoint. A more relevant viewpoint would be to say this: the lure of JBoss lies in its popularity. To date, it has been the dominant open source & free J2EE implementation. This has been its only distinguishing point - open source, free, J2EE. With Jonas gaining ground, with Geronimo coming out of the starting gates like greased lightning, and with the loss of the CDN developers on JBoss' side, and its very public pissing off of the Jetty people, the dominant role of JBoss in the future has been rightfully called into question. At the same time that viable contenders are coming into the public spotlight, JBoss' ability to meet this challenge has been more questioned than ever. And if alot of people start buying into those alternatives, JBoss the app server could quickly sink below the waves - precisely because J2EE makes it so easy to port away from a loser. And if JBoss the app server starts doing that, we can all guess what will happen to JBoss Group LLC.

    \MF\
    4- OPEN SOURCE IS THE ONLY KNOWN STRUCTURE TO WITHSTAND FULL FRONTAL ASSAULT ON A MARKET BY MICROSOFT.
    \MF\

    I think you miss the poster's point here entirely. The J2EE market is setup so that it's not a death-blow to the market if one or more indivudal companies go under the waves. So long as the market exists, you can always jump ship to another app server if you have to. Microsoft has done in individual companies, but they've failed _miserably_ at demolishing non-Microsoft standards.

    \MF\
    5- everyone else has dissapeared through MASSIVE consolidation that is still going on. We haven't, in fact we are one of the reasons for that consolidation. The safe choice may become JBoss very soon.
    \MF\

    Holy shit, IBM and BEA have disappeared? Oh my god!!! When did this happen?

    Jonas is gone? Geronimo has halted all development? Sun has filed for bankruptcy? This is terrible - we'd all better run to Marc Fleury LLC right now!!! He's the _only man_ who knows how to beat Microsoft.

    \MF\
    6- We will make some announcements re: results and finances of JBoss inc. You will be pleasantly surprised about our financial viability (if you are rooting for us that is :)
    \MF\

    Give us a break - JBoss Group LLC is a privately held company. You can announce anything you want and no one has any way to determine the veracity of those announcements.

    The best indicator I've seen so far of your financials is that you had to start the "JBoss J2EE Founders Program" (http://www.jboss.org/services/press/j2eefounders.pdf) because JBoss Group LLC apparently lacks the money to do full J2EE verification itself.

    \MF\
    [On lock-in in open source]

    That is false, and a myth coming from open source FUD litterature.
    \MF\

    This is hardly a myth. Having source code available is no guarantee. The fact is, a company can easily lock themselves into a non-standard open source code base, and find suddenly that development on that code base has gone fallow. Sure, maybe they could hire someone to take up the gauntlet - but at the same time, they watch competing products rush in with new features and fixes while their open source code base just sits there, with no movement.

    Here's an exercise for you. Imagine Sybase's RDBMS was open source. Let's say the company released it that way as a last-ditch effort to gain market share. Would you take this Sybase open source over something like Oracle on the pay side, or Postgres on the open source side? Most people wouldn't - they'd see Sybase as a dead end, open source or not.

    An open source product is only truly valuable if it has development momentum behind it. Lose that momentum, and people migrate away from it, even if it's painful and they are locked in. Frankly, in my view, JBoss is rapidly losing its momentum just as competitors are gaining it in a big way.

        -Mike
  122. JBoss claims that J2EE is dead[ Go to top ]

    Marc Fleury,
    I agree that Open Source is better than ANY vendor sofware, including IBM/BEA/MS.

    I also do not think Mike S. will ever be a client of jBoss, but quite a few others will.

    .V
  123. JBoss claims that J2EE is dead[ Go to top ]

    Bill,

    > ROTFL! You're just scared that we're killing you in the market. Don't worry, Marc has continually made you offers over the years, so you'll have some job security once the layoffs start. ;-)
    >

    Yes, I used to think Cedric would be good in the persistence layers and god knows we needed help there. Since then the hibernate wagon came along and so we hired Gavin instead :)

    The best developers do live in open source...

    marcf
  124. Core competency[ Go to top ]

    [..] you start seeing specs become bloated and tailored to individual vendors, or take years (J2EE 1.4) to come out with a final release, see open source and proprietary products come out to fill out the numerable holes in the spec [..]


    This is precisely the point I was trying to make above. An open source project should not just follow the herd and implement every spec on the planet. It really can write its own ticket.

    > Our microkernel is a strong implementation to rival even the other containers you talk about. We handle component depencies, life cycle, configuration, classloading, hot deployment, and have robust connector architecture to provide remoteness, security and transactions, and finally have an abstraction implementation so that developers can write plugins to deploy their own types of components and get automatically the features I described above. What we haven't done is package and productize our microkernel.

    For what it's worth, I think you should go ahead and do that. Then you can advertise the fact that by ditching EJBs companies can slash their development prices.

    > It's sad that all we get from this JBoss 4.0 roadmap on TSS is non-constructive babble Why not ask some constructive questions like "HOw does MicroJBoss compare to other lightweight containers and IoC?" "What will it take to implement JSR-88?" "What exactly has been implemented for web services so far?"

    Usually a project should speak for itself. And there is not a lot of good will towards JBoss because there is a widespread perception that some people in your group are *only* in it for the money, and also because of hyperbolic, unreliable statements, from the project's leaders. Therefore it is viewed with the same cynicism that one feels towards BEA and IBM. I think that's why you are not getting the same warm fuzzy feeling as might be felt by Linus Torvalds, Alan Cox, Eric Raymond, etc.
  125. Core competency[ Go to top ]

    Our microkernel is a strong implementation to rival even the other containers you talk about. We handle component depencies, life cycle, configuration, classloading, hot deployment, and have robust connector architecture to provide remoteness, security and transactions, and finally have an abstraction implementation so that developers can write plugins to deploy their own types of components and get automatically the features I described above. What we haven't done is package and productize our microkernel.

    >
    Guglielmmo Lichtner wrote:
    > For what it's worth, I think you should go ahead and do that. Then you can advertise the fact that by ditching EJBs companies can slash their development prices.
    >
    not sure if this a poke or not, but I'll assume not

    Since our product is free and our support contracts cover all JBoss projects, this really isn't much of factor, IMO. We're into providing alternatives and choices. AOP and POJO based development is an intriguing alternative IMNSHO.

    > > It's sad that all we get from this JBoss 4.0 roadmap on TSS is non-constructive babble Why not ask some constructive questions like "HOw does MicroJBoss compare to other lightweight containers and IoC?" "What will it take to implement JSR-88?" "What exactly has been implemented for web services so far?"
    >
    > Usually a project should speak for itself. And there is not a lot of good will towards JBoss because there is a widespread perception that some people in your group are *only* in it for the money, and also because of hyperbolic, unreliable statements, from the project's leaders. Therefore it is viewed with the same cynicism that one feels towards BEA and IBM. I think that's why you are not getting the same warm fuzzy feeling as might be felt by Linus Torvalds, Alan Cox, Eric Raymond, etc.

    We are guilty of wanting to form a company and I think this is where a lot of the hostility resides. I believe that we are stronger as a company than as a collection of independent consultants. That a strong company helps our open source projects and strong open source projects help our company. Its a good synergy IMO. Personally I've never been happier as my intellectual, idealistic, and financial goals are all wrapped up in one employment position. We will not succeed if it is all about the money.

    Bill
  126. Core competency[ Go to top ]

    For what it's worth, I think you should go ahead and do that. Then you can advertise the fact that by ditching EJBs companies can slash their development prices.

    > >
    > not sure if this a poke or not, but I'll assume not

    Not a "poke". I really do mean that J2EE development costs (hours of work) are way too large, and that you are in a unique position to slash development prices by pushing new, simpler ideas.

    You can do it, because you don't have to keep sixteen thousand vendors happy at the same time.
  127. Generalized containers[ Go to top ]

    This is precisely the point I was trying to make above. An open source project should not just follow the herd and implement every spec on the planet. It really can write its own ticket.

    >

    Agreed but let me repeat that fine line. We need both though. I think innovation beyond the spec is what is driving us from a pure technology standpoint. It is painfully obvious to me though anytime I do a customer talk that spec compliance is important from risk minimization standpoint.

    Therefore, for our innovation to gain more acceptance, it is important that JBoss inc drives innovation in JSR's. SUN just invited us to join the JCP and we did. We want to apply en-masse to JSR's in 2004. Sacha is driving it.

    Just EJB3.0 for example with hints of "interceptors" would resemble the portable interceptor stuff from the corba days and is one of the reasons for JBoss' popularity in ISV. That would be a good thing I think.
     
    >
    What we haven't done is package and productize our microkernel.
    >
    > For what it's worth, I think you should go ahead and do that. Then you can advertise the fact that by ditching EJBs companies can slash their development prices.
    >

    Yes on the productization, it is already used that way and need more marketing.

    The dual line EJB/noEJB is confusing to experts only. Newbies tend to fall squarely on one side, they either love EJBs because they think it is expert stuff and therefore they are valuable or they hate it because they read the rants of some clown.

    I believe they are 2 different marketing messages. JBoss at heart really is a microkernel, AOP interceptors and services. The EJB container notion is a high level construct that enforces a certain semantic for components. EJB is a particular instance of a generalized container. Generalized containers are the future of containers (lightweight or not). While historically the focus at JBoss has been on EJB and interceptors for it, the work that has been going on for the past 3 year also has been a build out of the framework that allows US to ASSEMBLE that container. 3.x series exposed all interceptors, which we would like to see standardized in EJB 3.0. The full framework is what is exposed in 4.0 as a raw construct.

    I believe these customer driven stacks will live in TELCO designs, in embedded systems, in next generation assembly lines machines. Already MCI uses us in production on a clustered system. We hear vodafone has selected JBoss over BEA for next generation designs. AMD uses JBoss to power their lights-out fab lines (the ones that build chips) with a microkernel based grid control. Yeah this is the robot control stuff. HERE WE ARE FAR AWAY FROM THE J2EE WEB SERVER FLUFF... this is the land of microkernels, automated grids and controls and service oriented architectures, but like for real :) we in the "middleware is everywhere" world already. Thanks IBM for pumping some $200M in marketing that message in the USA.

    Generalized containers is not for everyone though :) I still believe pre-cooked, pre-assembled containers as EJB (read blue from www.jboss.org if you haven't :) is the way to go for a large majority of developers and these are seen as complex? J2EE IS ready for an overhaul. Transparent middleware may be the way.

    >
    some people in your group are *only* in it for the money, and also because of hyperbolic, unreliable statements, from the project's leaders.
    >

    COME ON...

    on the hyperbolic statements, sue me. I am the son of a marketer, marketeer at heart. In fact if you read SUN's FUD about JBoss in the field it says that "Marc Fleury was JUST a systems engineer while at SUN" which is completely false since I wasnt' even a systems engineer, I was a regular pre-sales engineer. Scratch that, I was PHONE support SE for my first 3 month :).

    On the money. I am tired of trying to justify ourselves between the communists that can't stand us because we are actually making money at open source and the hyper capitalists that called us fools in the beginning for giving away our code... You know what? screw the extremists, there is a pragmatic reality where professional open source is a working formula.

    Do you really think I would have started in open source if I was that greedy? he he, dude, I was making a killing in 1999 in the silicon valley as a consultant, and put my stash of money to work in JBoss. That is how I funded the initial company that put JBoss on the map.

    JBoss has a reputation for cutting edge techno extremists. It is something I am personally very proud of even if it doesn't help in sales. The guys around me are better at code than I ever was. At the same time building a real company out of the open source movement is the most exciting experience for us. Everybody in the company has this focus. I am annoyed when I read "the curse of the enginneer" on this very site. It is a pathetic rant on the lack of respect for the engineer in the US. You want respect? you go get respect! COME ON get OFF YOUR ASS and start something. In the words of the "godfather", power isn't given it is taken. So go and take it.

    I always viewed open source, and more recently professional open source, as an empowerement of the engineer trade. All of the sudden any engineer can reach masses thanks to the net, no marketing dollars needed, no distribution hurdles. DO you realize the potential? do you? We are witnessing the power of the net in JBoss, dears. It is just systematic organization of the open source power. It is available to any of you, so GET OFF YOUR ASSES and start something instead of whinning.

    >
    Therefore it is viewed with the same cynicism that one feels towards BEA and IBM. I think that's why you are not getting the same warm fuzzy feeling as might be felt by Linus Torvalds, Alan Cox, Eric Raymond, etc.
    >

    he he he, dude you are new to this open source game right? eric raymond is more controversial than me, I am a school girl compared to the amount of heat that guy takes and dishes. He's got balls I am almost in awe of him.

    Linus? have EVER read posts by linus? I did, I studied the style. He can be the most abrasive when he wants. There is a post where he TRASHES some guys at SGI for "soiling the purity of the code" and calling them pigs or something to that effect. And yes I am also in awe of him. The only thing I regret is that he didn't start his company to prove "engineers can do it". Right now he is working for the man at OSDL (largely IBM funded) which is a great position for him. I read in WIRED that he did make a killing on RH/VA stock. Good for him.


    marcf
  128. Generalized containers[ Go to top ]

    \MF\
    I believe they are 2 different marketing messages. JBoss at heart really is a microkernel, AOP interceptors and services.
    \MF\

    No, no, and yes - I guess 1 out of 3 ain't bad, though. Of course, since you keep endlessly repeating the same things over and over again, true or not some people will believe the message. Even though your own publically available source completely contradicts it.

    \MF\
    [...]this is the land of microkernels, automated grids and controls and service oriented architectures, but like for real :) we in the "middleware is everywhere" world already. Thanks IBM for pumping some $200M in marketing that message in the USA.
    \MF\

    How many tens of megabytes does your "micro" kernel require on startup, Marc? How many classes are required?

    \MF\
    On the hyperbolic statements, sue me. [and blah blah blah]
    \MF\

    Potential users won't sue you. They'll just outright stop listening to you, and move on to people who make sense. Even though the rhythmic chanting works for a long time, it doesn't work forever. Eventually people _do_ notice that you never, ever come close to delivering on any of the chants. What's worse, is that the hyperbole doesn't stop with you, but seems to have affected the technical people as well - which is I believe what the original poster was speaking to. I'd say the majority of people don't take most of your technical leaders seriously anymore, for the simple reason that their over-the-top claims just don't match the code they're producing.

    As for the other 5 or 6 posts you've put out in machine-gun fashion - all I can say is, lay off the Antwerpian pipe, man. There's a fine line between a visionary and a ranting lunatic who more drug and sex references than relevant ones about their company....

         -Mike
  129. Generalized containers[ Go to top ]

    (ERRATA: In my previous post I wrote the message like three times and I did not mean to say that I work at BEA - I do not work there.)

    > on the hyperbolic statements, sue me. I am the son of a marketer, marketeer at heart. In fact if you read SUN's FUD about JBoss in the field it says that "Marc Fleury was JUST a systems engineer while at SUN" which is completely false since I wasnt' even a systems engineer, I was a regular pre-sales engineer. Scratch that, I was PHONE support SE for my first 3 month :).

    Well, that certainly clears some things up for me.

    > On the money. I am tired of trying to justify ourselves between the communists that can't stand us because we are actually making money at open source and the hyper capitalists that called us fools in the beginning for giving away our code... You know what? screw the extremists, there is a pragmatic reality where professional open source is a working formula.

    I just don't see that. If you want to consult, then do it, but why consult on a single product? Maybe I should see JBoss taking over the world, but I don't. As far as I can see, the situation right now is that most companies use JBoss to play with or to develop with, but deploy on WLS or WebSphere. I don't have hard numbers, but I see the managers' attitude in corporate America. I am not even sure if you are certified yet.

    > Do you really think I would have started in open source if I was that greedy? he he, dude, I was making a killing in 1999 in the silicon valley as a consultant, and put my stash of money to work in JBoss. That is how I funded the initial company that put JBoss on the map.

    I guess you do it because you want to work for yourself. I can understand that.
     
    > JBoss has a reputation for cutting edge techno extremists.

    Mostly you have the reputation of a guy who thinks he knows it all and who thinks everybody else knows nothing. You should feel good - it's what marketing people are often like.

    > You want respect? you go get respect! COME ON get OFF YOUR ASS and start something. In the words of the "godfather", power isn't given it is taken. So go and take it.

    I feel pretty respected. You actually thought I was Cedric!

    > he he he, dude you are new to this open source game right? eric raymond is more controversial than me, I am a school girl compared to the amount of heat that guy takes and dishes. He's got balls I am almost in awe of him.

    I doubt he insults people like you do. And as far as I can see he maintains his projects without bragging about it to everyone.

    > Linus? have EVER read posts by linus? I did, I studied the style. He can be the most abrasive when he wants. There is a post where he TRASHES some guys at SGI for "soiling the purity of the code" and calling them pigs or something to that effect. And yes I am also in awe of him.

    I don't you are anything like him.
  130. Generalized containers[ Go to top ]

    \\MF

    > Linus? have EVER read posts by linus? I did, I studied the style. He can be the most abrasive when he wants. There is a post where he TRASHES some guys at SGI for "soiling the purity of the code" and calling them pigs or something to that effect. And yes I am also in awe of him.
    \\MF


    \Guglielmo Lichtner\
    I don't you are anything like him.
    \Guglielmo Lichtner\

    Some people are abrasive - open source or not. Some are even outright rude.

    People like Torvalds and Raymond have been known to be quite rude at sometimes, and certainly can be abrasive. _But they come up with the goods_. Like them or don't, either way you have to _respect_ them, because they've got the technical goods. I may disagree with a Torvalds, even vehemently, but I always know his opinion is grounded very solidly in the designs and in code. When an outline is made by various pieces of the Linux kernel team, they do it. People who do solid work, sometimes even stellar work, who keep to their words, and who are above all _honest_ about their work garner respect.

    The case with JBoss people is entirely different - more often then not, claims are based on nothing but hot air. Some other claims are flatly contradicted by the code. Goals are elucidated - and the code that's delivered doesn't match the claims. And honesty? Puh-lease. I've seen some very abrasive open source people admit they've been wrong in public many times, along the lines of "shit, this was a judgement call, and I completely blew it - Joe Schmoe was right, and his code proves it". How often do you see a JBoss person do this? Answer: they don't. When they are proven wrong beyond a shadow of a doubt, the most honorable thing they do is sweep it under the rug and don't mention it.

    Ultimately, people trust someone like Torvalds because he engenders trust. You know he'll give the straight scoop, and that he's proven he can deliver. JBoss Group, on the other hand, does not engender _any_ kind of trust in me. Even their own brand of brashness and forthrightness aren't the hallmarks of geniuses who don't suffer fools, it's not someone giving you straight talk - its alot closer to arrogance, a taste for the vulgar, and valuing the warm glow of publicity over honesty. A 14 year old using nasty words because he discovers he can. You know when Linus cusses you out, there's cause. When Bill Burke or Marc Fleury cusses you out, more often it's out of petty cruelty, insecurity, or bravado.

        -Mike
  131. Generalized containers[ Go to top ]

    The case with JBoss people is entirely different - more often then not, claims are based on nothing but hot air.


    I always suspected this, but I wasn't sure because I never took the time to read the code. I would bet that they still don't have TCP/IP multiplexing. But I hope I am wrong - I mean - after all these years.

    If you can give a good summary of what you know about the real JBoss, I'd love to hear to it. My impression so far is that

    a) It's basically compliant with J2EE (99% say)

    b) It scales about as well as Tomcat.

    But I have never used it - haven't needed to so far. Marketing-wise, I would trust Jonas more because they don't push it. The more a product gets pushed, the more it means it *needs* to be pushed.
  132. Generalized containers[ Go to top ]

    \Guglielmo Lichtner\
    If you can give a good summary of what you know about the real JBoss, I'd love to hear to it.
    \Guglielmo Lichtner\

    The raw basics for J2EE are there. Problems arise in some areas of performance, and also in error detection and recovery.

    The JMS implementation has always been pretty bare bones. It functions well enough for simple things, but isn't really suitable for any advance uses or where robustness or speed are desired. The classic example of this is in its XA support - they claim to support XA, but the recover() call returns new Xid[0]. What this means is that if server running JMS is recycled for any reason, it just loses all transactions which were in flight at the time. There are also many missing extras that just about every other implementation I know of has, such as dead message queues and the like. People who need JMS and choose JBoss quite often use another JMS provider. Even Bill Burke has bitched about this, I believe in the JMS development forum.

    I've read just about every line of their JMS system, and I can tell you definitively it's not very good. They claim to be working on a total JMS re-write, in two pieces. One piece is a robust JMS, which they claim they can do in 1 man month. They haven't been able to do it in years, but suddenly they can do it one man month. No emphasis at all on performance/scalability. The other leg is a peer-to-peer JMS system which looks interesting in its very early form, but this has very low priority.

    New AOP/Aspect work was done almost exclusively by Bill Burke. A home-grown AOP system was cobbled together (as opposed to using one of the many existing AOP implementations), which is missing features of just about every other AOP system out there. A number of "aspects" were written by Bill as well. I've read the code for this stuff in detail as well. There's no design for it, the docs are one web page, and there's lots of hard coding of behavior to lock you into one model only. Obviously, there are also bugs, since this is early work, but bug reports on these go unanswered. More recently Bela Ban and Ben Wang have been doing aspect work with caches. Their own AOP-cache, TreeCacheAOP, has silently replaced Bill Burke's own TxCache. Overall this is a good thing, since TxCache had hard-coded behavior in how it dealt with transactions, and was buggy to boot (e.g. if you entered a transaction and tried to read a field out of a cached object, you be an NPE). Bela has turned this work over to Ben Wang so that Bela can go off and make JGroups easy to use for people who don't know what a multi-homed system is. In the end, you have 95% of all of their Aspect code written by Bill Burke in a hacking binge over a month and a half. This is what they're pinning their future on. As an aside on that, I've tracked down a grand total of 5 people who have even bothered to try the new AOP stuff, and all of them have reported signifcant issues and problems that will be fixed "real soon now". This is a good thing, since the AOP system eats memory for lunch, as does each of the "aspects". Good thing TxCache is gone, 'cuz it tended to multiply the memory use of a given POJO by 10 times per object or more (in some cases, it was using 4-5 objects _per field_ in a POJO, even primitives).

    On general support of XA transactions by the container, once again we have a transaction system with no transaction log. Cycle JBoss with in-flight transactions for any reason, and those in-doubt threads locking up big pieces of your database will just hang there 'till the universe dies from heat death.

    Clustering is a very murky subject. JBoss people claim fabulous things for their clustering code. Funny thing is, try to find someone who actually uses JBoss clustering in production. I've only found a handful, and the bulk of them are doing fairly vanilla HTTP session replication. Look into the underlying code, and you see alot of information coming in which isn't checked for validity, a number of exceptions which are just caught and printed out and never dealt with, and a mountain of unanswered cries for help for problems on the cluster forum.

    In my experience, the code which seems the most solid is the code written by the dear departed members of JBoss - various CDN folks, Rickard, etc. Gavin and Bela's code seems to be a cut above the current JBoss crop. I can't speak too much on Gavin's side, but Bela has solid designs, and most of his code looks reasonable. JGroups (aka JavaGroups), the basis of JBoss clustering, does have a distressing tendency to also eat critical exceptions, and some alarming "ToDo" comments in the code. I'm involved in a long term evaluation of this (lots of little tests over the course of many months when I find the time), so eventually I'll have a better handle on all of this.

    JBoss folks of course have never been too keen on me - can't blame them for that. But they turned downright nasty when I started looking at their code in detail and pointing out flaws. I've pointed out various problems them with great specificity, and no JBoss contributor has ever acknowledged any of them as problems, or that anything will be done about them. Bill still has his NPE in his TxCache, but fortunately that's going away if Ben can complete TreeCacheAOP (I wonder how much Bela has actually contributed - 95% of the TreeCache code in total in CVS was checked in by Ben Wang). The JMS XA recover() still returns an empty Xid, but I'm supposed to believe some JBoss genius will re-write their entire JMS implementation including a robust transaction log in 1 man month. Several bugs in clustering have been reported in just the past few weeks, but no JBoss person will come out in public and admit that their clustering may not be quite bulletproof. We're supposed to buy into their JBoss-lock-in AOP and "J2EE a la carte" aspect system - but other than TreeCacheAOP it's all written by Bill Burke in spare moments in bursts of hacking. Nothing has been done with it in literally months. We're supposed to believe in ultra fast and robust and feature-rich clustered caching, but the lead on this is off twiddling network interfaces. Gavin King is doing all sorts of things to use Hibernate all over the place within JBoss. Gavin seems to have great abilities, and hasn't yet been captured by the JBoss system, but he reports only 5% done at this point.

    This list is by no means exhaustive, but I believe it shows some trends rather well. Why don't people believe MarcF and BillB? Because the code doesn't match their mouths.

         -Mike
  133. Generalized containers[ Go to top ]

    Thanks for the download.

    > I've read just about every line of their JMS system, and I can tell you definitively it's not very good. They claim to be working on a total JMS re-write, in two pieces. One piece is a robust JMS, which they claim they can do in 1 man month.

    I think they have about 25 days left now.

    > On general support of XA transactions by the container, once again we have a transaction system with no transaction log. Cycle JBoss with in-flight transactions for any reason, and those in-doubt threads locking up big pieces of your database will just hang there 'till the universe dies from heat death.

    I get the picture. With no recovery log, can't expect much in this area ..
     
    > Clustering is a very murky subject. JBoss people claim fabulous things for their clustering code.

    I have always been suspicious of such claims.

    > Funny thing is, try to find someone who actually uses JBoss clustering in production. I've only found a handful, and the bulk of them are doing fairly vanilla HTTP session replication.

    They probably just use clusters of machines that talk to each other. Now with Bela's new cache they can copy over finished transactions from machine to another, but that's not that useful (to me) because it treats intra- and inter-process transactions pairs on different footings.

    > In my experience, the code which seems the most solid is the code written by the dear departed members of JBoss - various CDN folks, Rickard, etc. Gavin and Bela's code seems to be a cut above the current JBoss crop. I can't speak too much on Gavin's side, but Bela has solid designs, and most of his code looks reasonable. JGroups (aka JavaGroups), the basis of JBoss clustering, does have a distressing tendency to also eat critical exceptions, and some alarming "ToDo" comments in the code. I'm involved in a long term evaluation of this (lots of little tests over the course of many months when I find the time), so eventually I'll have a better handle on all of this.

    I know that Bela did his post doc in Ken Birman's dept. (the grandad of group communication,) so he knows his stuff. However, I have serious doubts about his protocol stack implementation: each protocol layer uses two threads: one for sending events down and one of sending events up (e.g. messages, joins, suspicions, etc.) This means that if your protocol stack has 5 layers, each message can only be delivered after 5 context switches. Unfortunately this really eats into the performance. And since the CPU is busy switching, at full blast the processor starts dropping messages (for example in ring-based total ordering,) and the flow control scales down the throughput to a few hundred messages a second. This is based on personal hands-on experience but not with JavaGroups, so this is just a supposition. Maybe Bela knows something I don't.

    JavaGroups is really designed to experiment with new protocols and see how they behave, but I think it would take some work to make it speed up. But again, maybe Bela can say different. I almost used it in production once, and ended up going with Ensemble (a predecessor written in ocaml), and Bela also thought that Ensemble was better for production. It was four years ago, but the architecture doesn't seem to have changed at all.

    > Several bugs in clustering have been reported in just the past few weeks,

    Thanks for letting me know that ..

    > Gavin King is doing all sorts of things to use Hibernate all over the place within JBoss. Gavin seems to have great abilities, and hasn't yet been captured by the JBoss system, but he reports only 5% done at this point.

    I was pretty suprised to see Gavin working for Marc. I mean, he's already put out Hibernate - he's got nothing to prove. I hope the spirit of JBoss doesn't infect Hibernate because to date they really have been run using two very very different approaches, especially to performance. Hibernate seems to be very focused on performance.

    This is also the first time I see an acquisition happen between two open source projects ..

    > This list is by no means exhaustive, but I believe it shows some trends rather well. Why don't people believe MarcF and BillB? Because the code doesn't match their mouths.

    You actually verified this by looking at the code. As for myself, when I see the unlikely claims, I smelled a rat, so I decided I probably didn't need to look at it.

    Let's see how these other projects go.
  134. Generalized containers[ Go to top ]

    Hi Guglielmo,


    > JGroups (aka JavaGroups), the basis of JBoss clustering, does have a
    > distressing tendency to also eat critical exceptions

    Where ? I'd appreciate you telling me, so I can fix them.

    > and some alarming "ToDo" comments in the code.

    Where ? What is alarming ?



    > However, I have serious doubts about his protocol stack implementation: each
    > protocol layer uses two threads: one for sending events down and one of
    > sending events up (e.g. messages, joins, suspicions, etc.)

    Same design as Ensemble. Same design as Matt Welsh's (highly praised on TSS) SEDA architecture.

    Note that you can define (for each protocol) whether you want both threads, or just the up_thread, or just the down_thread, or *none*. In the latter case, you can eliminate threads altogether. Have a look at my default_minimalthreads.xml config.


    > This means that if
    > your protocol stack has 5 layers, each message can only be delivered after 5
    > context switches.

    Not really true: depends on your configuration.

    > Unfortunately this really eats into the performance. And since the CPU is busy
    > switching, at full blast the processor starts dropping messages

    This is an unsubstantiated claim. The default stack is used in many production environment, and I've never seen it lose messages.

    > (for example in ring-based total ordering,)


    That protocol was not written by me, and is not the default config (e.g. in JBoss). TOTAL has not really been tested thoroughly, so it may lose messages. But please don't claim JGroups is losing messages, when in fact one (untested) config does. Again, the default config is very reliable.


    > and the flow control scales down the throughput to a few hundred messages a
    > second.

    A few hundred msgs/sec is actually pretty good. Have a look at http://www.jgroups.org/javagroupsnew/docs/performance.html, where I compare performance for 10K messages in a 4-node cluster (2 senders). I get sustained throughput of 350 msgs/sec, that's 3.5MB/sec. That's slightly slower than Spread, but Spread is written in C !


    > JavaGroups is really designed to experiment with new protocols and see how
    > they behave, but I think it would take some work to make it speed up.

    Not true. I improved performance massively ca 2 years ago. Also, memory consumption has gone down. Please have another look at JGroups.

    > But again, maybe Bela can say different. I almost used it in production once, > and ended up going with Ensemble (a predecessor written in ocaml), and Bela
    > also thought that Ensemble was better for production. It was four years ago,
    > but the architecture doesn't seem to have changed at all.

    Yes - that was true several years ago. It is *not* true any longer. As a matter of fact, I'm thinking of dropping Ensemble support because I don't use/support it anymore.

    Regards,

    Bela
  135. Generalized containers[ Go to top ]

    Hi Guglielmo,


    Hi, thanks for dropping in.

    > > JGroups (aka JavaGroups), the basis of JBoss clustering, does have a
    > > distressing tendency to also eat critical exceptions
    >
    > Where ? I'd appreciate you telling me, so I can fix them.

    This wasn't me saying it - it was Mike. You'll have to take it up with him.
    Mike, where did you see this stuff?
     
    > > and some alarming "ToDo" comments in the code.
    >
    > Where ? What is alarming ?

    As above.

    > > However, I have serious doubts about his protocol stack implementation: each
    > > protocol layer uses two threads: one for sending events down and one of
    > > sending events up (e.g. messages, joins, suspicions, etc.)
    >
    > Same design as Ensemble. Same design as Matt Welsh's (highly praised on TSS) SEDA architecture.

    Perhaps I am wrong in this. I really don't know the ocaml code because it's in ocaml and I don't want to read it. However, I did some experiments by myself just to learn and I think you take a huge performance hit from context switching.

    To me it's just math, really. Assuming a workload can be done in parallel, if you use N cpus to run N + M threads (say), you get less thruput than if you run only N threads. The missing thruput ends in the kernel's scheduler. The only reason for running N + M threads would be to support N + M concurrent tasks, so that they all make progress together. You get better latency, but the thruput goes down.
     
    > Note that you can define (for each protocol) whether you want both threads, or just the up_thread, or just the down_thread, or *none*. In the latter case, you can eliminate threads altogether. Have a look at my default_minimalthreads.xml config.

    Will do if I need to use it. Thank you for clearing that up. Actually, what I really want to know is if the protocols that are *already* there really need those threads.

    But you are saying that the extra threads actually speed things up, right?

    > > This means that if
    > > your protocol stack has 5 layers, each message can only be delivered after 5
    > > context switches.
    >
    > Not really true: depends on your configuration.

    If you can bring an example here it would be great.
     
    > > Unfortunately this really eats into the performance. And since the CPU is busy
    > > switching, at full blast the processor starts dropping messages
    >
    > This is an unsubstantiated claim. The default stack is used in many production environment, and I've never seen it lose messages.

    I did not mean to say that the messages don't get delivered - you retransmit them - but the flow control should automatically scale down the window when messages are dropped, right?

    I am not making this up, because unlike you I did not do this in school. But this is basically the central point of the UCSB totem protocol. The good performance lies in the flow control.

    > > (for example in ring-based total ordering,)
    >
    >
    > That protocol was not written by me, and is not the default config (e.g. in JBoss). TOTAL has not really been tested thoroughly, so it may lose messages. But please don't claim JGroups is losing messages, when in fact one (untested) config does. Again, the default config is very reliable.

    Indeed. That came out wrong - JavaGroups does not lose messages. That's what reliable multicast is all about. I have no knowledge of any bugs anywhere. I was trying to make a point about how cpu utilization determines maximum thruput.

    > > and the flow control scales down the throughput to a few hundred messages a
    > > second.
    >
    > A few hundred msgs/sec is actually pretty good. Have a look at http://www.jgroups.org/javagroupsnew/docs/performance.html, where I compare performance for 10K messages in a 4-node cluster (2 senders). I get sustained throughput of 350 msgs/sec, that's 3.5MB/sec. That's slightly slower than Spread, but Spread is written in C !

    I think an old client of mine is using a two-node cluster with Ensemble, and when I set it up I tried a quick test and it did 5,000/s.

    I also tried my own pure-java implementation of just totem (stripped-down, bare-bones, no stack basically) and it also did 5000/s. This is on fast ethernet. We are talking total ordering. But the first time I wrote I used threads everywhere and it was performing in the hundreds.

    > > JavaGroups is really designed to experiment with new protocols and see how
    > > they behave, but I think it would take some work to make it speed up.
    >
    > Not true. I improved performance massively ca 2 years ago. Also, memory consumption has gone down. Please have another look at JGroups.

    Okay.

    > > But again, maybe Bela can say different. I almost used it in production once, > and ended up going with Ensemble (a predecessor written in ocaml), and Bela
    > > also thought that Ensemble was better for production. It was four years ago,
    > > but the architecture doesn't seem to have changed at all.
    >
    > Yes - that was true several years ago. It is *not* true any longer. As a matter of fact, I'm thinking of dropping Ensemble support because I don't use/support it anymore.

    Ok. It was nice to get your take on this stuff.
  136. Generalized containers[ Go to top ]

    However, I did some experiments by myself just to learn and I think you take a

    > huge performance hit from context switching.

    I guess the newer JDKs (1.4) are really good at context switching thousands of threads.

     
     
    > Will do if I need to use it. Thank you for clearing that up. Actually, what I
    > really want to know is if the protocols that are *already* there really need
    > those threads.


    Well, for some of the newer protocols, I have hard-coded that it doesn't need an up_thread or down_thread or both. But for most protocols, you can configure this.

     
    > But you are saying that the extra threads actually speed things up, right?

    Yes - concurrency increases throughput as well. Also, for some protocols you *need* threads, because they would block otherwise. Won't go into too much details though.


      
    > > Not really true: depends on your configuration.
    >
    > If you can bring an example here it would be great.


    Just look at default-minimalthreads.xml. In the most extreme tuning, you'll end up having the channel thread serving 1 message all the way down to the transport, and vice versa (although that's not highly performing).

    But what I'm saying is that you can really fine-tune the thread usage to your application, and your network properties.



    > I did not mean to say that the messages don't get delivered - you retransmit
    > them - but the flow control should automatically scale down the window when
    > messages are dropped, right?

    Yes that's right (if you use the FC protocol, that is). There is no flow control in the default protocol stack config, because most people never send hundreds of messages/sec.


    > I am not making this up, because unlike you I did not do this in school. But
    > this is basically the central point of the UCSB totem protocol. The good
    > performance lies in the flow control.


    Yes. Note that Spread is an implementation of Amir's total token protocol, and they get 541 10 k msgs/sec, whereas JGroups get 451 10k msgs/sec.

      
    > Indeed. That came out wrong - JavaGroups does not lose messages. That's what
    > reliable multicast is all about. I have no knowledge of any bugs anywhere. I
    > was trying to make a point about how cpu utilization determines maximum thruput.

    Okay, that's what I thought you meant. Just wanted to set it straight, so people don't start thinking JGroups is not reliable.


    > > A few hundred msgs/sec is actually pretty good. Have a look at http://www.jgroups.org/javagroupsnew/docs/performance.html, where I compare performance for 10K messages in a 4-node cluster (2 senders). I get sustained throughput of 350 msgs/sec, that's 3.5MB/sec. That's slightly slower than Spread, but Spread is written in C !
    >
    > I think an old client of mine is using a two-node cluster with Ensemble, and when I set it up I tried a quick test and it did 5,000/s.


    Do you have more info on this ? What's the msg size here ?

    Note that I send 10k messages. I could easily send smaller-sized messages, and pack multiple smaller msgs into one 10k message, to achive similar results.
    Note that 10k is not the frag size (at least on the system I tested on), so I could probably increase the size and *not* send more messages.

    Also note that the new transport I've been mentioning, will include (besides spraying multiple interfaces):

    #1 Logical addresses. Stay with the member as long as it is alive (VM lifetime). This will greatly ease the effects of shunning (being expelled and redmitted to a group under a new identity).

    #2 High performance transport. Using a separate queue for each send and recv socket. I identified some bottlenecks in {Multicast,Datagram}Socket.send(), plus receive().

    I'm anxious to see the results for the performance tests with the new transport.
     
    Bela
  137. Generalized containers[ Go to top ]

    However, I did some experiments by myself just to learn and I think you take a

    > > huge performance hit from context switching.
    >
    > I guess the newer JDKs (1.4) are really good at context switching thousands of threads.

    Could be.

    > Well, for some of the newer protocols, I have hard-coded that it doesn't need an up_thread or down_thread or both. But for most protocols, you can configure this.

    Interesting.

    > > But you are saying that the extra threads actually speed things up, right?
    >
    > Yes - concurrency increases throughput as well. Also, for some protocols you *need* threads, because they would block otherwise. Won't go into too much details though.

    I must disagree that concurrency increases thruput. Once the degree of concurrency reaches a certain limit, adding more decreases it. If you have a one-cpu machine, with total ordering, this means you'll get maximum throughput with one thread. I am assuming the thread doesn't block on i/o operations.

    I just don't see how increasing context switching can increase thruput no matter what - it's more time spent in the kernel and less spent in user space. It decreases the latency, but it doesn't increase the thruput.

    > But what I'm saying is that you can really fine-tune the thread usage to your application, and your network properties.

    But like you said it also depends on the design of the code. Depending on the design you may *need* up- and down- threads.

    > > I did not mean to say that the messages don't get delivered - you retransmit
    > > them - but the flow control should automatically scale down the window when
    > > messages are dropped, right?
    >
    > Yes that's right (if you use the FC protocol, that is). There is no flow control in the default protocol stack config, because most people never send hundreds of messages/sec.

    So, out of curiosity, what's the largest thruput you have seen reported in a real app? If I had to use JGroups I would definitely want to set it up so it performs well at full blast.

    > Yes. Note that Spread is an implementation of Amir's total token protocol, and they get 541 10 k msgs/sec, whereas JGroups get 451 10k msgs/sec.

    I think I misunderstood before - I thought you were using one-kilobyte messages - not 10K. I have used 1k in my own implementation in the past, because that's what they used in the totem article.

    > Okay, that's what I thought you meant. Just wanted to set it straight, so people don't start thinking JGroups is not reliable.

    Yup. Sorry about that one. I mean .. I hope it is reliable, but it's not easy to tell because with so many mini-protocols I find it hard to follow. I guess once you reach enough users that'll be the proof.

    > Do you have more info on this ? What's the msg size here ?

    As above, we are talking about two different message sizes - 1Kb for me. Hence the difference. Big oops on my part there.

    > Note that I send 10k messages. I could easily send smaller-sized messages, and pack multiple smaller msgs into one 10k message, to achive similar results.
    > Note that 10k is not the frag size (at least on the system I tested on), so I could probably increase the size and *not* send more messages.

    If you more details of your test setup I would love to hear them so I can use them as a rule of thumb.

    Did your test include total-ordering?

    > Also note that the new transport I've been mentioning, will include (besides spraying multiple interfaces):
    >
    > #1 Logical addresses. Stay with the member as long as it is alive (VM lifetime). This will greatly ease the effects of shunning (being expelled and redmitted to a group under a new identity).
    >
    > #2 High performance transport. Using a separate queue for each send and recv socket. I identified some bottlenecks in {Multicast,Datagram}Socket.send(), plus receive().
    >
    > I'm anxious to see the results for the performance tests with the new transport.

    Me too.
  138. Generalized containers[ Go to top ]

    \Guglielmo Lichtner\
    I must disagree that concurrency increases thruput. Once the degree of concurrency reaches a certain limit, adding more decreases it. If you have a one-cpu machine, with total ordering, this means you'll get maximum throughput with one thread. I am assuming the thread doesn't block on i/o operations.

    I just don't see how increasing context switching can increase thruput no matter what - it's more time spent in the kernel and less spent in user space. It decreases the latency, but it doesn't increase the thruput.
    \Guglielmo Lichtner\

    Well, see what Bela said on this "Also, for some protocols you *need* threads, because they would block otherwise. Won't go into too much details though.".

    The most straightforward models for doing threading and I/O in Java involve many situations where blocking can happen. Even when you're not necessarily blocked on I/O, the easiest algorithms often involve blocking a thread for something.

    If you take this approach, then what Bela says makes sense - in such a design you can end up with many threads blocked, and more concurrency equals more threads equals better over all throughput. Most of your threads are sitting around not doing much :-)

    In addition - if you a scenario where do have many blocking threads, you can end up in a situation where at un-blocking time you have CPU spikes of processing. Here, the CPU can show modest use, and even if you end up "overdriving" the CPU, sometimes the kernel thread scheduler can win out over optimizing your own code (e.g. the scheduler will slice your work more efficiently than you can do yourself, even with the overhead of switching thread contexts).

    However - if you are in this situation you can hit an ultimate wall that won't go away. Ultimately, the costs do catch up, and I believe this is where you're coming from Guglielmo. In Bela's published tests, various optimizations pushes him up to ~400-700 msgs/sec (the best shown is 692 msgs/sec for 2 senders of 500 bytes each).

    What's missing here is a bit of a reality check - 700 msgs/sec is pretty lousy on any kind of modern hardware on a 100MBit Ethernet network. The results should be an order of magnitude better, in the several thousands of messages per second. There is clearly a bottleneck here. Some possible causes:

      - Tiny burst sizes. Not enough time to make meangingful measurements.
      - Excessive blocking and use of synchronous request-response network calls
      - Exhaution of some resource, most likely CPU in this case if we're talking a 100MBit ethernet.

    My own guess would be that a combination of tiny burst size and use of lots of objects that get serialized are the most likely culprits. Remove unnecessary seriliazation and make the burst sizes something reasonable (500 would be a bare-bones minimum) and then see where you are. At that point, then threading issues may come into play - context switches don't come into play until you're already very fast, and we're not yet at the very fast rate.

    To correlate to some of my own tests, using NIO selectors and worker pools for reading I/O, writing I/O, and processing results....if I use a model where I dispatch to a thread for reading, do work, and then write out in another thread, and then cycle back to the Selector, I peak out around 1,900 messages per second. Some of this time is code executing to get from the Selector popping out to me doing real work with it, the rest is Selector overhead and thread context switches.

    When I switch the model to a "sticky" one, where I do N reads and request executes in a single thread (where N is configurable, and we always pop out if there's no more data coming in or we can't write out w/out blocking - 10 seems to be a very nice "N" number for me...), then my message rate bounces up to 2,800 msgs/second. When heavy "bursts" of traffic are coming in, this sticky model avoids Selector and context switch overheads for the majority of messages, and yet still yields gracefully at a configurable point, or when its about to block.

    In my example, the code was already pretty fast. I'm already going at well over 1000 msgs/second, which means that sub-millisecond times now count and have an impact. At this point, I make a compromise between I/O multiplexing and its overhead, and reducing latency for bursts - and I find that Selector overhead and thread context overhead has become a "larger" factor than it ever was for my object serialization-based protocol.

    In that sort of serialization protocol, you're often going to max out well below 1,000 messages per second because you're spending a tremendous amount of time in ObjectInputStream and ObjectOutputStream. With that sort of cost weighing you down, you're not even going to really notice thread context switching times. In my own tests, it's abundantly clear that it's quicker to send an object's bytes "over-the-wire" on a 100MBit Ethernet network than it is for one side to serialize it and another side to deserialize it. It sounds weird, but in this case the network is faster than code.

         -Mike
  139. Spread says....[ Go to top ]

    Here's what the Spread web site says on the subject (http://www.spread.org/docs/spreadoverview.html):

    "Spread is optimized and can handle over 8,000 1Kbytes messages a second in local area network settings".

    This is precisely in the ballpart for I'd expect a C/C++ implementation, a highly optimized Java implementation (better then I can do - harrumph!) to do on a 100MBit LAN. This is 1600% better throughput than is reported on the JGroups page :-/

         -Mike
  140. Husker Doo![ Go to top ]

    Bela, I just checked in on adapttcp because the TCP/IP numbers just seemed too low, and I saw the following:

         out=new DataOutputStream(sock.getOutputStream());

    and:
         in=new DataInputStream(sock.getInputStream());

    Have you perhaps considered changing this to:

         out=new DataOutputStream(new BufferedOutputStream (sock.getOutputStream()));
         in=new DataInputStream(new BufferedInputStream (sock.getInputStream()));

    You may see a very small difference there.....

          -Mike
  141. Husker Doo![ Go to top ]

    Hi Mike,

    changed it, and the perf is about the same. I currently send the msgs sequentially, probably parallelizing (1 sender q per TCP send socket) will help.

    Still surprisingly low numbers for TCP...
    Bela


    > Bela, I just checked in on adapttcp because the TCP/IP numbers just seemed too low, and I saw the following:
    >
    > out=new DataOutputStream(sock.getOutputStream());
    >
    > and:
    > in=new DataInputStream(sock.getInputStream());
    >
    > Have you perhaps considered changing this to:
    >
    > out=new DataOutputStream(new BufferedOutputStream (sock.getOutputStream()));
    > in=new DataInputStream(new BufferedInputStream (sock.getInputStream()));
    >
    > You may see a very small difference there.....
    >
    > -Mike
  142. Generalized containers[ Go to top ]

    Well, see what Bela said on this "Also, for some protocols you *need* threads, because they would block otherwise. Won't go into too much details though.".


    This might have some something to do with the RpcProtocol class. I guess the protocols are easier to implement correctly, but they don't perform as well. Bela still claims that adding threads SPEEDS things up, but I disagree. If he runs his tests on 4-cpu boxes, that could be why (a layer on each cpu, basically.)

    I wrote my implementation of totem as a regular state machine, all asynchronous stuff. It's actually easier that way because the article I was following gives the protocol exactly in that format. Plus I used the State design pattern, which is not half bad.
     
    > If you take this approach, then what Bela says makes sense - in such a design you can end up with many threads blocked, and more concurrency equals more threads equals better over all throughput. Most of your threads are sitting around not doing much :-)

    Right, except it's already costing you something to do the switching. But in the end it's a quantitative question. It's possible that I am chasing a wild goose and it doesn't add up to much.
     
    > In addition - if you a scenario where do have many blocking threads, you can end up in a situation where at un-blocking time you have CPU spikes of processing. Here, the CPU can show modest use, and even if you end up "overdriving" the CPU, sometimes the kernel thread scheduler can win out over optimizing your own code (e.g. the scheduler will slice your work more efficiently than you can do yourself, even with the overhead of switching thread contexts).

    But with total ordering on 1-cpu boxes, more than one thread still means less thruput. It's basically an all serial workload - extra parallelism just takes away cpu cycles. I actually tried this. If I find some time I'll dig up my old test and check it over again.

    > However - if you are in this situation you can hit an ultimate wall that won't go away.

    I am not sure what this means :) It sounds like management :) No, seriously, do you mean when your thruput starts to plateau?

    > Ultimately, the costs do catch up, and I believe this is where you're coming from Guglielmo. In Bela's published tests, various optimizations pushes him up to ~400-700 msgs/sec (the best shown is 692 msgs/sec for 2 senders of 500 bytes each).

    I assume we are still talking 10kb-messages. That would be about 56Mbs. That sounds pretty good to me. But if the box has four cpus ...

    > What's missing here is a bit of a reality check - 700 msgs/sec is pretty lousy on any kind of modern hardware on a 100MBit Ethernet network. The results should be an order of magnitude better, in the several thousands of messages per second. There is clearly a bottleneck here. Some possible causes:
    >
    > - Tiny burst sizes. Not enough time to make meangingful measurements.
    > - Excessive blocking and use of synchronous request-response network calls
    > - Exhaution of some resource, most likely CPU in this case if we're talking a 100MBit ethernet.

    Usually low thruput comes from flow control. There's a "multicast storm" going on, especially without total ordering, where everybody is sending things at the same time. The buffers get filled up, so the window size drops (I used the old Jacobson algorithm,) and it's very difficult to reach the theoretical limit of the network (i.e. 100Mbps full-duplex, I guess.) And the buffers get filled up because the CPU can't keep up.

    With gigabit ethernet, specifically with 1) jumbo frames (9kb) and 2) interrupt coalescing, things should improve. But one also has to remember that at point the project in question may be able to afford infiniband (about $2,000 per node) and just do synchronous RDMA.

    > My own guess would be that a combination of tiny burst size and use of lots of objects that get serialized are the most likely culprits. Remove unnecessary seriliazation and make the burst sizes something reasonable (500 would be a bare-bones minimum) and then see where you are. At that point, then threading issues may come into play - context switches don't come into play until you're already very fast, and we're not yet at the very fast rate.

    I don't know much about the bursting problem. With a token protocol the system is very well behaved because the flow control is built into the token.

    > In my example, the code was already pretty fast. I'm already going at well over 1000 msgs/second, which means that sub-millisecond times now count and have an impact. At this point, I make a compromise between I/O multiplexing and its overhead, and reducing latency for bursts - and I find that Selector overhead and thread context overhead has become a "larger" factor than it ever was for my object serialization-based protocol.

    Yes! Thank you, Sir.

    > In that sort of serialization protocol, you're often going to max out well below 1,000 messages per second because you're spending a tremendous amount of time in ObjectInputStream and ObjectOutputStream. With that sort of cost weighing you down, you're not even going to really notice thread context switching times.

    I agree. In my own code I wrote my own basic serialization. Then I wrote a stream interface on top of it (symmetrically replicated InputStream.) But at least I was able to do a test without serialization mixed in.

    With Java you really have to be careful because performance problem pop up all over the place.
  143. Generalized containers[ Go to top ]

    \Guglielmo Lichtner\
    This might have some something to do with the RpcProtocol class. I guess the protocols are easier to implement correctly, but they don't perform as well. Bela still claims that adding threads SPEEDS things up, but I disagree. If he runs his tests on 4-cpu boxes, that could be why (a layer on each cpu, basically.)
    \Guglielmo Lichtner\

    Your second sentence is bang-on - generalized stacks don't tend to do as well in performance. For something intended as a low-level "product" that typically will have many app layers on top of it, I prefer a double-approach: a generalized interface for adding in truly generic and unanticipated components, or to ease development of components where performance doesn't matter, and rampant incestuousness between very well-known components who purposely know the guts of the rest of the low-level system. This violates all sorts of good-engineering principles, but can unleash really big performance gains. Many people scoff at this and spout all sorts of OO hogwash - but as I said, I'm already in the zone of sub-millisecond, and reaching into private bits can lower code paths by a couple of hundred microseconds, leading to a hundred or more msgs/sec.

    In any case - adding threads may speed up Bela's case because his design may inherently have alot of internal thread blocking built in. See the following bit from you...
     
    \Guglielmo Lichtner\
    I wrote my implementation of totem as a regular state machine, all asynchronous stuff. It's actually easier that way because the article I was following gives the protocol exactly in that format. Plus I used the State design pattern, which is not half bad.
    \Guglielmo Lichtner\

    Bingo! This is precisely what my own code does - all reads and writes use a state machine, allowing for partial reads/writes that can be "suspended" and picked up at a later time. To some extent the higher level code also follows a state machine, for example for timeouts and the like, but I haven't done this as aggressively as it could be. This is part of the reason why I max out at 2,800 msgs/sec.

    Bela is hogtied a bit in this area, in that multicast sockets don't have a usable nonblocking read/non-blocking write facility. This doesn't mean that the higher level protocols can't be async and non-blocking, but some I believe are because it fits the model the best.

    \Guglielmo Lichtner\
    Right, except it's already costing you something to do the switching. But in the end it's a quantitative question. It's possible that I am chasing a wild goose and it doesn't add up to much.
    \Guglielmo Lichtner\

    Yes, it is quantitative. Ultimately, it's a question of average thread context switch times in microseconds, time spent in synchronous/blocking "idle time", and your own processing overhead. In JGroups case there is definitely some blocking idle time, and also the processing overhead may be more than you'd think. In that sort of world the microseconds for thread context switches look just like noise - why count thread countext switch microseconds, when you've got 1 millisecond+ times to consider?

    \Guglielmo Lichtner\
    But with total ordering on 1-cpu boxes, more than one thread still means less thruput. It's basically an all serial workload - extra parallelism just takes away cpu cycles. I actually tried this. If I find some time I'll dig up my old test and check it over again.
    \Guglielmo Lichtner\

    This is assuming optimum efficiency outside of the CPU (e.g. I/O is kept streaming in, no extended blocking). But imagine if the CPU is sitting idle alot of the time, and is only bursting now and again? That's a whole 'nuther kettle of fish.

    I understand what you're talking about - you use a state machine and really optimized ordering lists of some kind and you just zip along. This is one design - harder to do, but probably the fastest on all counts. Another design is to replace some (or all!) of the state machine with threads, method calls, and mutexes. Less efficient, but easier to do.

    \Guglielmo Lichtner\
    > However - if you are in this situation you can hit an ultimate wall that won't go away.

    I am not sure what this means :) It sounds like management :) No, seriously, do you mean when your thruput starts to plateau?
    \Guglielmo Lichtner\

    Yes, sorry I wasn't more specific. What I've found is that the less efficient your code is (and "efficient" here includes blocking areas, not just code speed), the more you benefit from parallelism to show higher throughput.

    Here's an example: let's say you've got to disk force stuff to a slow disk and it takes 50 milis to do so (it's a really slow disk :-). The disk force is a blocking point, and for 1 thread you can do a request every 50 millis (plus some overhead). Let's say you get smart and implement disk-batching. Each disk force still takes 50 millis, but you can share that 50 millis across threads by forcing for a number of people. So 1 guy forces in 50 millis + X, where X is overhead for that. 2 Guys might now force in 50 millis + 2X, 4 in 50 millis + 4x, etc. Leaving overhead out for a moment, you'll see a thruput of 1 every 50 milliseconds...then 2 every 50 milliseconds, then 4 every milliseconds - latency is constant (except for that bit of overhead), but throughput increases.

    The problem here is that you can only do this so much. If you try to "batch" too many threads on a disk force, you'll find that the batch-requests are too spread out, and you start waiting for more time for "batchees" than it takes to do the force itself (e.g. to batch 100 guys you may need to wait 500 millis to get that many requests). Ultimately, somewhere in there you see a wall: you can only batch up so many things on the lock.

    In any case, the programmer of this system "sees" that more concurrency gets him better throughput - really, he's getting more efficiency use out his serialized blocking on this slow disk. But there's still going to be a wall.

    Now take that slow disk, throw it onto the FDR, and buy a fast RAID array and a fast controller. Now your disk forces might go down to 5 milliseconds. This order of magnitude drop in serialized time makes a huge difference. The old non-batcher was doing maybe 20 forces a second. The batcher on the old disk maybe popped you up to 100 forces a second, by amortizing that cost over multiple threads (an average of 5 of them). But now, with your zippy array, you can do almost 200 forces a second w/out even resorting to batching.

    If you go with a state-machine, non-blocking, highly-async and resumable-partial-completion design for messaging, you're making your system act like the one with the zippy hard drive. The few "bottlenecks" that are there have small enough timespans that optimizing something like thread switching is important, because your bottleneck time is on the same order as thread context switching time. In one case the "bottleneck" is removed by buying a new disk, in a case like ours the bottleneck is removed by eliminating unnecessary blocking - the code is executing whenever it needs to be, and is quieiscient only if there's nothing to do, and we've optimized code paths so that our overhead is on the same order of thread-switching overheads. On max loads, your CPU is kept busy all the time even with a tiny number of threads.

    But if you have alot of blocking threads by design, the CPU is just sitting around doing nothing the majority of the time. Adding more threads makes sense, because if you're idle for 5 or 6 milliseconds at a clip, the microseconds of a thread context switch aren't going to matter.

    \Guglielmo Lichtner\
    I assume we are still talking 10kb-messages. That would be about 56Mbs. That sounds pretty good to me. But if the box has four cpus ...
    \Guglielmo Lichtner\

    "Test #1" from the JGroups page uses 500 byte objects. It gets 453 msgs/sec for 1 sender, 692 msgs/sec for 2 (presumably, this means 346 messages per second per sender). We're talking 226KB/sec and 346KB/sec respectively - which is just awful for a 100MBit network.

    Clearly, there's a bottleneck here, and it's not the network - subsequent tests seem to have steady message/sec rates, but increasing KB/sec (the max is 2968, a 10 times increase in KB/sec but a _lower_ msg/sec rate of 296, this is "Test #3"). The bottleneck is in how many messages/sec, not KB/sec, JGroups can push out.

    \Guglielmo Lichtner\
    Usually low thruput comes from flow control. There's a "multicast storm" going on, especially without total ordering, where everybody is sending things at the same time. The buffers get filled up, so the window size drops (I used the old Jacobson algorithm,) and it's very difficult to reach the theoretical limit of the network (i.e. 100Mbps full-duplex, I guess.) And the buffers get filled up because the CPU can't keep up.
    \Guglielmo Lichtner\

    I don't think 1 sender is causing a multicast storm :-) In the case of JGroups, JGroups isn't coming anywhere near the theoretical network limit of 100MBps, it's peaking around 97MBps below that. What you say is true if you're keeping the NIC streaming with data as best you can, but clearly JGroups isn't doing that at all.

    \Guglielmo Lichtner\
    I don't know much about the bursting problem. With a token protocol the system is very well behaved because the flow control is built into the token.
    \Guglielmo Lichtner\

    It's a simple problem - let's say you have 1,000 connections. 9,950 are pretty much quiescent, 49 have light traffic, and 1 has a burst of traffic coming in on it. If you take the route of [wait for I/O event] -> [handle request/dispatch] -> put I/O conn back on Selector queue], then your "bursting" connection is going to have to go through the "wait/handle/put back in wait queue" cycle. But if you optimize the "handle" portion, you can avoid the select call overhead and dispatching overhead (this assumes you're using a pool of worker threads). In fact, you avoid non-Kernel context switches altogether for the sticky "N" messages you process. The "price" of this is that the other 49 light traffic connections will be slightly starved, but for many applications this is OK.

    \Guglielmo Lichtner\
    I agree. In my own code I wrote my own basic serialization. Then I wrote a stream interface on top of it (symmetrically replicated InputStream.) But at least I was able to do a test without serialization mixed in.

    With Java you really have to be careful because performance problem pop up all over the place.
    \Guglielmo Lichtner\

    Absolutely true. You have to be very careful in Java to really understand what's going on, what's blocking, what object creation's happening, etc. I really like the NIO stuff because it looks like the first Java sub-system to really take this to heart.

         -Mike
  144. Generalized containers[ Go to top ]

    [..] and reaching into private bits can lower code paths by a couple of hundred microseconds, leading to a hundred or more msgs/sec.


    What type of applications do you work on, out of curiosity?

    > Bela is hogtied a bit in this area, in that multicast sockets don't have a usable nonblocking read/non-blocking write facility.

    I am doing non-blocking reads from my MulticastSocket though. I just set a timeout. That was for jdk1.3. Now with jdk1.4 I should switch to select, I guess.

    > This is assuming optimum efficiency outside of the CPU (e.g. I/O is kept streaming in, no extended blocking). But imagine if the CPU is sitting idle alot of the time, and is only bursting now and again? That's a whole 'nuther kettle of fish.

    Yes. My cpu is about 80%. If I put a two-node ring on the same box the thruput actually goes up slightly, by about 10%, looks like. So you have a point. But when you figure the added cpu load from a real online transaction processing application, there won't be any idle cpu.

    > I understand what you're talking about - you use a state machine and really optimized ordering lists of some kind and you just zip along.

    I am not sure what you mean by really optimized ordering lists, do you mean for ordering the messages? I did write my own (because performance was bugging me,) I think I am recycling the links in the list. But I don't think I am recycling the message objects themselves.

    > Yes, sorry I wasn't more specific. What I've found is that the less efficient your code is (and "efficient" here includes blocking areas, not just code speed), the more you benefit from parallelism to show higher throughput.

    Yes. The problem with threads is really with pre-emptive multitasking, where you switch just because the time is up, not because you are not runnable any more.

    > Here's an example: let's say you've got to disk force stuff to a slow disk and it takes 50 milis to do so (it's a really slow disk :-). The disk force is a blocking point, and for 1 thread you can do a request every 50 millis (plus some overhead). Let's say you get smart and implement disk-batching. Each disk force still takes 50 millis, but you can share that 50 millis across threads by forcing for a number of people. So 1 guy forces in 50 millis + X, where X is overhead for that. 2 Guys might now force in 50 millis + 2X, 4 in 50 millis + 4x, etc. Leaving overhead out for a moment, you'll see a thruput of 1 every 50 milliseconds...then 2 every 50 milliseconds, then 4 every milliseconds - latency is constant (except for that bit of overhead), but throughput increases.

    And then the thruput levels off for a while and the latency keeps going up. And then after a long while the thruput actually decreases ..

    > In any case, the programmer of this system "sees" that more concurrency gets him better throughput - really, he's getting more efficiency use out his serialized blocking on this slow disk. But there's still going to be a wall.

    Right.

    > If you go with a state-machine, non-blocking, highly-async and resumable-partial-completion design for messaging, you're making your system act like the one with the zippy hard drive.

    But the point that's relevant to me personally is that I am interested in total ordering. So you can't actually do writes or reads in parallel. They are all done in a sequence. The disk example works great because the workload is parallelizable to a very large degree. It's Amdahl's Law. You can only optimize away the parallel part of the workload, which in my case is minimal.

    > "Test #1" from the JGroups page uses 500 byte objects. It gets 453 msgs/sec for 1 sender, 692 msgs/sec for 2 (presumably, this means 346 messages per second per sender). We're talking 226KB/sec and 346KB/sec respectively - which is just awful for a 100MBit network.

    With the provisor that I don't remember Bela saying this (or maybe you are basing this on your own benchmark?), yes, it's not good. If you turn on total ordering that means the thruput would come out to 346/s * 500, like you said. With my own implementation I am getting (I just dug it up again) 4,200/s * 1024. But again, I am not so sure that Bela is getting such low numbers.
     
    > I don't think 1 sender is causing a multicast storm :-) In the case of JGroups, JGroups isn't coming anywhere near the theoretical network limit of 100MBps, it's peaking around 97MBps below that. What you say is true if you're keeping the NIC streaming with data as best you can, but clearly JGroups isn't doing that at all.

    When I said that I didn't realize that the thruput was so low. I really couldn't tell you what it's doing at this point. Serialization plus threads?

    > Absolutely true. You have to be very careful in Java to really understand what's going on, what's blocking, what object creation's happening, etc. I really like the NIO stuff because it looks like the first Java sub-system to really take this to heart.

    Do you happen to know if the non-blocking reads in java.nio are more efficient than doing a read() with timeout like in jdk1.3. I have a feeling that it doesn't make much difference, because it still needs to check. I don't mean for handling a large number of connections. I mean for checking for incoming multicast packets.
  145. Generalized containers[ Go to top ]

    ///
    > I am doing non-blocking reads from my MulticastSocket though. I just set a timeout. That was for jdk1.3. Now with jdk1.4 I should switch to select, I guess.
    ///

    No you can't: the NIO impl doesn't support MulticastSockets. There is no MulticastSocket.open().

    Bela
  146. Total ordering[ Go to top ]

    Guglielmo: But the point that's relevant to me personally is that I am interested in total ordering. So you can't actually do writes or reads in parallel. They are all done in a sequence. The disk example works great because the workload is parallelizable to a very large degree. It's Amdahl's Law. You can only optimize away the parallel part of the workload, which in my case is minimal.

    Guglielmo, I am curious why you are so interested in total ordering. For most tasks, a synchronous total ordering protocol logically introduces negative (not "decreasing," but "negative") scale for even the second server, let alone third, fourth, etc. And making the protocol asynchronous largely negates the benefits of total ordering. I'm sure there is a use case you're thinking about, but I'm missing it. It seems that you either want ease of distributed development (synchronous total ordering provides simplicity in state transitions) or you accept the complexity of non-total-ordering in order to achieve scalability.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Clustered JCache for Grid Computing!
  147. Generalized containers[ Go to top ]

    ////
    > I must disagree that concurrency increases thruput. Once the degree of concurrency reaches a certain limit, adding more decreases it. If you have a one-cpu machine, with total ordering, this means you'll get maximum throughput with one thread. I am assuming the thread doesn't block on i/o operations.
    ///

    I don't disagree. I said one has to find the right balance. That's why you can (a) turn off (almost) all threads in JGroups, (b) turn all of them on (2 threads/protocol), or (c) use a combination.

    I don't know your app. For example, in a J2ME app you may want to reduce the number of threads and context switches. I don't offer a one-size-fits-all mechanism, but you can tune it extensively.


     
    > > > I did not mean to say that the messages don't get delivered - you retransmit
    > > > them - but the flow control should automatically scale down the window when
    > > > messages are dropped, right?


    No. I implemented a credit based flow control system. This is based on a paper from Birman et al.



    > So, out of curiosity, what's the largest thruput you have seen reported in a real app? If I had to use JGroups I would definitely want to set it up so it performs well at full blast.


    I'm going to get those numbers in the next couple of months (I now have access to HP's labs in Cupertino, great !).


    > If you more details of your test setup I would love to hear them so I can use them as a rule of thumb.


    Well, what's missing from the URL I posted ?

    Note that these tests originate from a consortium of European universities and companies in the framework of a project called ADAPT. I didn't write those tests, but just got the src code from them and ran it again a flow-control version of JGroups, plus Spread. They benched JGroups, Spread and some impl called JBora. That's also why you have those strange burst rates and sleeps in there. I didn't change their tests at all.


     
    > Did your test include total-ordering?


    No. Both of our total order protocols (sequencer and token) need more work.

    Cheers,

    Bela
  148. Generalized containers[ Go to top ]

    No. I implemented a credit based flow control system. This is based on a paper from Birman et al.


    The UCSB guys believe that the token flow control is more effective. But I don't have the authority to agree or disagree with them. I would be curious to see a comparison.

    > I'm going to get those numbers in the next couple of months (I now have access to HP's labs in Cupertino, great !).

    Cool. What's the hardware like there? Itanium? Just x86? PA-RISC?

    > Well, what's missing from the URL I posted ?

    You are right. It's there. The total messages are not a lot, but maybe you addressed that below.

    > Note that these tests originate from a consortium of European universities and companies in the framework of a project called ADAPT. I didn't write those tests, but just got the src code from them and ran it again a flow-control version of JGroups, plus Spread. They benched JGroups, Spread and some impl called JBora. That's also why you have those strange burst rates and sleeps in there. I didn't change their tests at all.

    Thanks for the reference.

    > > Did your test include total-ordering?
    > No. Both of our total order protocols (sequencer and token) need more work.

    And based on your experience do you have an opinion on how total ordering should affect the results?
  149. Toekn based flow control[ Go to top ]

    I changed the subject...


    > The UCSB guys believe that the token flow control is more effective. But I
    > don't have the authority to agree or disagree with them. I would be curious to
    > see a comparison.


    I don't know. What I don't like about a rotating token-based impl of total order is that you always have traffic with the token roation going on, even if you don't send anything. Also, you'll end up needing to adjust the token rotation time *dynamically* because fixed intervals are always a waste (either too high or too low).

     
    > > I'm going to get those numbers in the next couple of months (I now have access to HP's labs in Cupertino, great !).
    >
    > Cool. What's the hardware like there? Itanium? Just x86? PA-RISC?


    - Network: 100MB, 1GB switched ethernet
    - Hardware: everything :-) Proliant 4CPU, HP
    - OS: Linux, HPUX, Windows, etc
    - VMS: SUN, JRockit etc
    - Up to 20 physical boxes... (I'm starting with 4 though)


    I'm drooling ...

     

    > > > Did your test include total-ordering?
    > > No. Both of our total order protocols (sequencer and token) need more work.
    >
    > And based on your experience do you have an opinion on how total ordering should affect the results?

    Can't say. I'm not really interested right now, because nobody uses total order. Most people okay with FIFO.

    I definitely plan to take a look at our total order protocols at some point, but for now it is low prio.

    Bela
  150. Toekn based flow control[ Go to top ]

    I don't know. What I don't like about a rotating token-based impl of total order is that you always have traffic with the token roation going on, even if you don't send anything. Also, you'll end up needing to adjust the token rotation time *dynamically* because fixed intervals are always a waste (either too high or too low).


    You are right that there is always something being sent out. But that's what production applications are like. The network is there to be used. I think it's better to optimize the full-blast, steady state, than to be able to use the network for lots of different things at the same. Also, you can always add an extra network card on each machine and a hub.
  151. Toekn based flow control[ Go to top ]

    Guglielmo, I am curious why you are so interested in total ordering. For most tasks, a synchronous total ordering protocol logically introduces negative (not "decreasing," but "negative") scale for even the second server, let alone third, fourth, etc.


    I am not sure what you mean here by negative scale.

    > And making the protocol asynchronous largely negates the benefits of total ordering. I'm sure there is a use case you're thinking about, but I'm missing it.

    Well, for one, to do fault-tolerant corba you need atomic brodcast - total ordering basically. Any time you want to drive two programs in lock step, you need to agree on an order.

    Another application is a distributed lock manager. All locks must be created in the same order. Replicated database?

    Another possible application is gaming, e.g. LAN party :-) I don't play myself, but I am interested.

    But more generally I just like the semantics of the totem protocol. There's a well-defined order for all messages. Also, the token provides several functions in one, including ordering, retransmission, and flow control (and in my case, congestion control.) And the latency can be computed from the size of the ring.

    It's nice :-)

    No, seriously, as I said I don't have an application for it. But somebody else might, so after this discussion I am thinking that I should clean up this code and put it out there again. Maybe some startup with no money will use it for something.

    > It seems that you either want ease of distributed development (synchronous total ordering provides simplicity in state transitions) or you accept the complexity of non-total-ordering in order to achieve scalability.

    Before Totem (and the Transis project, I think) people believed that you could not get total ordering without a big performance loss. Certainly Birman believed this. However, when you look at Totem you clearly see that it is not the case.

    Anyhow, it's not like I am trying to force people to use it. I'm just geeking out here.
  152. Total Ordering[ Go to top ]

    Hi Guglielmo,

    I completely misunderstood you, and thus asked a stupid question. I thought you needed to find a total ordering protocol for a particular problem that you were working on. Instead, it seems that you are enjoying building one. By all means, enjoy! Sorry for the confusion.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Clustered JCache for Grid Computing!
  153. Toekn based flow control[ Go to top ]


    >
    > No, seriously, as I said I don't have an application for it. But somebody else might, so after this discussion I am thinking that I should clean up this code and put it out there again. Maybe some startup with no money will use it for something.
    >

    I am interested in testing what you have. if it is ok with you, please send me an email to talip at jpower dot org.

    thanks.
  154. Token based flow control[ Go to top ]

    Another application is a distributed lock manager. All locks must be created

    > in the same order.

    You can use FIFO. Then you have to use lock acquisition timeouts because of inherent deadlock danger. But, I agree, if you use total order, by ordering lock acquisition you can avoid deadlocks.


    > Replicated database?

    2PC works also on FIFO. But, if you have total order, some people claim you can reduce 2PC to 1PC in certain cases. The paper I'm referring to is the DRAGON project at ETHZ Switzerland.

    Bela
  155. Token based flow control[ Go to top ]

    You can use FIFO. Then you have to use lock acquisition timeouts because of inherent deadlock danger. But, I agree, if you use total order, by ordering lock acquisition you can avoid deadlocks.


    That's what I thought.

    > 2PC works also on FIFO. But, if you have total order, some people claim you can reduce 2PC to 1PC in certain cases. The paper I'm referring to is the DRAGON project at ETHZ Switzerland.

    Right. It must what Birman is referring to in his classic book, p. 469, section entitled "Advanced Replication Techniques":

    "What we can do is use [..] lightweight replication methods to replicate the volatile, cached database state within a group of database servers [..] Within this group, there is no need to run a multiphase commit protocol!"

    He goes on to explain that there is only one log. I guess it's the coordinator's log. But for availability it needs to be put on a shared disk.
  156. Token based flow control[ Go to top ]

    Another application is a distributed lock manager. All locks must be created

    > > in the same order.
    >
    > You can use FIFO. Then you have to use lock acquisition timeouts because of inherent deadlock danger. But, I agree, if you use total order, by ordering lock acquisition you can avoid deadlocks.
    >

    This is why I took the optimistic locking route in the TxCache. You only need to obtain hard locks in the prepare/commit phase to update objects and to test version ids. You can sort the locks before you request a distributed lock and thus avoid the overhead of total ordering as the locks will be obtained in a guaranteed order(because of the sort).

    Bill
  157. Token based flow control[ Go to top ]

    This is why I took the optimistic locking route in the TxCache. You only need to obtain hard locks in the prepare/commit phase to update objects and to test version ids. You can sort the locks before you request a distributed lock and thus avoid the overhead of total ordering as the locks will be obtained in a guaranteed order(because of the sort).

    >
    > Bill

    Bill, can you explain this a little bit more? how can learn more about tx with optimistic locking?

    thanks

    -talip
  158. Token based flow control[ Go to top ]

    This is why I took the optimistic locking route in the TxCache. You only need to obtain hard locks in the prepare/commit phase to update objects and to test version ids. You can sort the locks before you request a distributed lock and thus avoid the overhead of total ordering as the locks will be obtained in a guaranteed order(because of the sort).

    > >
    > > Bill
    >
    > Bill, can you explain this a little bit more? how can learn more about tx with optimistic locking?
    >
    > thanks
    >
    > -talip

    Not sure I understand what you want me to elaborate on. TxCache is retired and Bela/Ben Wang will be incorporating optimistic locking into TreeCache eventually.

    Is it that you want to understand more about optimistic locking and a distributed cache and how to avoid deadlock? Aren't you the guy with the Javaspaces implementation(JPower)?

    Bill
  159. Token based flow control[ Go to top ]

    Is it that you want to understand more about optimistic locking and a distributed cache and how to avoid deadlock? Aren't you the guy with the Javaspaces implementation(JPower)?

    >
    > Bill

    Hi Bill,

    JPower project is dead. It wasn't cluster aware so it didn't need distributed locking. Now I am working on something else which is running on top of a group communication software and now it requires distributed locking. I have never implemented distributed lock before. The plan is to gather as much information as I can before implementing. So it is very nice to have you guys here with tones of experience on this subject. I don't care about the 'cache' part of the discussion. I wanted to learn how you handled the locks when implementing TxCache since i want to avoid total ordering too.

    Regards,

    -talip
  160. Token based flow control[ Go to top ]

    This is why I took the optimistic locking route in the TxCache. You only need to obtain hard locks in the prepare/commit phase to update objects and to test version ids. You can sort the locks before you request a distributed lock and thus avoid the overhead of total ordering as the locks will be obtained in a guaranteed order(because of the sort).


    Does this mean that you roll transactions back as needed?

    If people don't mind coding retries that's okay. Perhaps JBoss has infrastructure for doing re-tries automatically. I know JMS does, but not every application is hooked up to JMS.
  161. Token based flow control[ Go to top ]

    This is why I took the optimistic locking route in the TxCache. You only need to obtain hard locks in the prepare/commit phase to update objects and to test version ids. You can sort the locks before you request a distributed lock and thus avoid the overhead of total ordering as the locks will be obtained in a guaranteed order(because of the sort).

    >
    > Does this mean that you roll transactions back as needed?
    >

    Yes, that was how it was coded.

    > If people don't mind coding retries that's okay. Perhaps JBoss has infrastructure for doing re-tries automatically. I know JMS does, but not every application is hooked up to JMS.

    It is quite trivial to add retries via an EJB interceptor or POJO advice. We actually do retries for Entity Bean deadlock, but not other scenarios at this time.

    Bela forwarded me the paper on how Total Ordering can avoid deadlocks. I thought Total Ordering was a bit too unscalable (totem sounds interesting, is it more scalable?).

    Bill
  162. Token based flow control[ Go to top ]

    It is quite trivial to add retries via an EJB interceptor or POJO advice. We actually do retries for Entity Bean deadlock, but not other scenarios at this time.


    I see. That's a very nice application of interceptors, aka ChainOfResponsibility pattern.

    > Bela forwarded me the paper on how Total Ordering can avoid deadlocks. I thought Total Ordering was a bit too unscalable (totem sounds interesting, is it more scalable?).

    If you mean scalable in terms of number of machines, it scales only to the extent that you can tolerate increased latency, because the token has to go around the ring. In the test I discussed above, with n = 3 you get 5ms. With n = 10 you probably get 15-16 ms, since it's proportional to n.

    Now, in my book 15 ms for a totally ordered message over 10 boxes is a bargain, but that's just me I guess.

    And the nice thing is that the thruput stays fixes as n increases. That's because each node receives the same messages, so the determining factor is how fast the (slowest) cpu can read the messages. In my case, that's 50Mpbs.

    There is also an extension of totem called the 'multiple ring protocol', with m rings and gateways, which scales better. But I haven't implemented it.
  163. Java network performance[ Go to top ]

    Hi Bela,

    Bela: A few hundred msgs/sec is actually pretty good. Have a look at http://www.jgroups.org/javagroupsnew/docs/performance.html, where I compare performance for 10K messages in a 4-node cluster (2 senders). I get sustained throughput of 350 msgs/sec, that's 3.5MB/sec. That's slightly slower than Spread, but Spread is written in C !

    On that linked page, you don't specify the hardware that the tests were run on. The Sun-Blade 1000s are pretty high-end workstations, taking up to 4x US3 CPUs, at speeds from 750Mhz up to well over 1Ghz. Not publishing the hardware details (other than the memory size of 512MB) makes the results somewhat hard to gauge, particularly since jGroups seems to be very CPU intensive, and with multiple threads it should be able to take advantage of multiple CPUs.

    If I read the report correctly, the 350 msgs/sec is actually 175 msgs/sec per sender, since you had two senders and you report the aggregate amount. Further, from your published numbers, the second server provided additional scalability over the first of 52%, 18% and 52% respectively (the three tables of results.) Since the total data going over the network with two senders was well under 50% of the network limit, what was limiting that scalability? Compare it to Spread, which accomplished 96% and 84% scalability (and a negative percentage on the last test, maybe because Spread was able to come closer to saturation of the network in that test with just one sender, although even in this test, the jGroups 2-sender numbers didn't even come close to reaching either the Spread 1-sender or 2-sender rate.)

    Anyway, I wouldn't actually choose to point this all out, except you say that "A few hundred msgs/sec is actually pretty good" and go on to say in the report that "The tests show that JGroups performs consistently below Spread.... This is to be expected, as Spread is written in C". I disagree with both of these statements. The problem is definitely not the language of implementation; other pure Java implementations of reliable multicast have shown sustained transfer rates and message frequency well over 2x of what you are showing, and accomplished it on a much smaller amount of hardware.

    So, as a starting point, it's OK to say "Here's where our performance is and we know it should be better and we're working on improving it and we have this test that will prove it over time," but make it a starting point, not a "Here's what we are limited to because of Java, but it is really good."

    Just my $.02. I think the message could be a lot more positive.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Clustered JCache for Grid Computing!
  164. Java network performance[ Go to top ]

    On that linked page, you don't specify the hardware that the tests were run on. The Sun-Blade 1000s are pretty high-end workstations, taking up to 4x US3 CPUs, at speeds from 750Mhz up to well over 1Ghz. Not publishing the hardware details (other than the memory size of 512MB) makes the results somewhat hard to gauge, particularly since jGroups seems to be very CPU intensive, and with multiple threads it should be able to take advantage of multiple CPUs.


    I did my tests at home on a hub, p4 1.5 Ghz, RDRAM 400Mhz, ancient eepro100 cards (82557 I think), linux2.4.* (no nptl - it was 2 years ago,) jdk1.3.*, just a clean-room implementation of totem (no jgroups). I think I can reproduce 5,000 1k-messages per second. For the record. I think it was two or three machines, everybody blasting messages continuosly.
  165. Java network performance[ Go to top ]

    On that linked page, you don't specify the hardware that the tests were run on. The Sun-Blade 1000s are pretty high-end workstations, taking up to 4x US3 CPUs, at speeds from 750Mhz up to well over 1Ghz.


    750MHz SunBlade 1000s with 1 CPU. I thought that SunBlades all come with the same config. My Sunblade 1000/Solaris 8 config is equivalent to an average PC you buy these days.

    Regards,
    Bela
  166. Java network performance[ Go to top ]

    750MHz SunBlade 1000s with 1 CPU. I thought that SunBlades all come with the same config. My Sunblade 1000/Solaris 8 config is equivalent to an average PC you buy these days.


    Okay, so it's one cpu. However, I do wonder about the size of the cpu cache. It might have an effect (but then again, it might not.) Don't Sun cpus have 8Mb of L2 or L3 cache?
  167. Java network performance[ Go to top ]

    Bela: 750MHz SunBlade 1000s with 1 CPU. I thought that SunBlades all come with the same config. My Sunblade 1000/Solaris 8 config is equivalent to an average PC you buy these days.

    Oh, it's yours? Lucky devil ;-) .. I'm still stuck with an old US2 for Solaris work. I don't know how they come configured nowadays; Sun only shows the Sun Blade 2000 model in the SDC hardware section, but it's 1-CPU.

    Out of curiousity, have you tested 1-CPU vs. 2-CPU systems? Since so much of this stuff (e.g. jGroups) is multi-threaded, I'm curious what deltas you see.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Clustered JCache for Grid Computing!
  168. Java network performance[ Go to top ]

    ///
    > I did my tests at home on a hub, p4 1.5 Ghz, RDRAM 400Mhz, ancient eepro100 cards (82557 I think), linux2.4.* (no nptl - it was 2 years ago,) jdk1.3.*, just a clean-room implementation of totem (no jgroups). I think I can reproduce 5,000 1k-messages per second. For the record. I think it was two or three machines, everybody blasting messages continuosly.
    ///

    - Impl of totem in Java or C ? (Java layer on top of a C runtime doesn't count :-))

    - What hardware, network ?

    Bela
  169. Java network performance[ Go to top ]

    - Impl of totem in Java or C ? (Java layer on top of a C runtime doesn't count :-))

    > - What hardware, network ?

    Okay, let's see if I can write this correctly for once ..

    You guys got me all excited about this last night so I dug up an old implementation of totem from a couple of years ago. At one point I had it on sourceforge, but I couldn't find any application for it. It's in pure java, absolutely no C code (I am not much a C coder ..)

    This is a test I did last night:

    machines 1 and 2:
    1.4 Ghz P4, 400Mhz RDRAM, 256Mb, linux 2.4.*, intel nic 82557 (ancient, I know,) e100 driver, jdk1.4.2

    machine 3:
    455 Mhz PIII, 128Mb SDRAM, linux 2.4.*, intel nic 82559, eepro100 driver, jdk1.4.0,

    Workload: three jvms sending packets at will and receiving each packet. This is totem, so at any given time only one jvm is sending obviously. Packet size: 1500 bytes (ethernet MTU). Number of messages received by any one vm per unit time: 4,200, or 6,152 Mb/s, or about 49Mbps. The token rotation time is about 5ms. The token rotation time is relevant to the latency, i.e. how long it takes to send a message.

    Interestingly, the useful load is not 6Mb/s, because each message has a 50-byte header, i.e. payload = 1450. That's 3% overhead.

    You might also be interested in knowing the window size: I set the maximum to 60, but it adjusts automatically to about 30. I put in a congestion control algorithm copied from the TCP (Jacobson) congestion algorithm, i.e. exponential back-off something or other. I put it in so I'd be able to telnet around my network while totem was running :-) Before I would start the test and then I couldn't use the network any more ...

    As far as differences in design from JavaGroups, I can list the following:
    1) only one thread per vm
    2) no serialization

    I never did anything with it because frankly I was mostly interested in making it go fast :) I never came up with a good application for it, so I let sit there. But it was fun optimizing it.
  170. Java network performance[ Go to top ]

    ///
    > Workload: three jvms sending packets at will and receiving each packet. This is totem, so at any given time only one jvm is sending obviously. Packet size: 1500 bytes (ethernet MTU). Number of messages received by any one vm per unit time: 4,200, or 6,152 Mb/s, or about 49Mbps. The token rotation time is about 5ms. The token rotation time is relevant to the latency, i.e. how long it takes to send a message.
    ///

    - What's a unit ? So you're multicasting (conceptually) packets (around the ring) and periodically count the number of packets received since the last count, divided by elapsed time since the last count ?

    - So everyone is sending and receiving ? Do you deliver msgs sent by yourself immediately ?

    - Did you run the 3 senders on different machines ? Interconnect is 100Mbps ?



    ///
    > You might also be interested in knowing the window size: I set the maximum to 60, but it adjusts automatically to about 30. I put in a congestion control algorithm copied from the TCP (Jacobson) congestion algorithm, i.e. exponential back-off something or other.
    ///

    Do you do retransmission of lost packets ? What do you do when (a) the token is lost due to crash of the member who has it while sending and (b) you have more than 1 token (e.g. network partition just healed) ?

    Bela
  171. Java network performance[ Go to top ]

    - What's a unit ?


    Several seconds. I can look up exactly what it is. Since it's doing 4,000/s you don't need a very large unit.

    > So you're multicasting (conceptually) packets (around the ring) and periodically count the number of packets received since the last count, divided by elapsed time since the last count ?

    Yes.

    > - So everyone is sending and receiving ?

    Yes.

    > Do you deliver msgs sent by yourself immediately ?

    Possibly. I don't remember but I can look it up. But it doesn't matter because you are not just counting your own messages. And all the processors are making progress together. So it really doesn't matter.
     
    > - Did you run the 3 senders on different machines ?

    Yes.

    > Interconnect is 100Mbps ?

    Yes. A 4-port hub. Cat5 cable, usual cheap stuff.

    > Do you do retransmission of lost packets ?

    Yes.

    > What do you do when (a) the token is lost due to crash of the member who has it while sending

    I am detecting the loss (token loss timeout) and making a new ring.

    > and (b) you have more than 1 token (e.g. network partition just healed) ?

    When you get a token from another ring (view) you make new ring.

    I do believe the membership protocol has a couple of bugs in it, but until I have a good application for this I probably won't fix it. There is also a RECOVERY state where you sync up the messages and you set up the "transitional configuration" and all that. I put in the code but I have no idea if it's correct because it's very hard to test. Frankly, I am not sure if EVS is better than VS.

    Anyhow, like I said I mostly wanted to see if Java would perform as needed.
  172. Total Token Protocol[ Go to top ]

    ///
    > I do believe the membership protocol has a couple of bugs in it, but until I have a good application for this I probably won't fix it. There is also a RECOVERY state where you sync up the messages and you set up the "transitional configuration" and all that. I put in the code but I have no idea if it's correct because it's very hard to test. Frankly, I am not sure if EVS is better than VS.
    >
    > Anyhow, like I said I mostly wanted to see if Java would perform as needed.
    ///

    Sounds like you invested a great amount of time in it !

    I recall we talked some time ago (1999/2000 ?) on the JGroups dev lists. Did you ever port your protocol to JGroups ?


    Bela
  173. Total Token Protocol[ Go to top ]

    Sounds like you invested a great amount of time in it !


    Yes. But most of the time I wasn't proceeding with much of a plan in mind. There's a guy who contacted me about it, so I am going to try to clean it up in the next few days so may be it can be used.
     
    > I recall we talked some time ago (1999/2000 ?) on the JGroups dev lists. Did you ever port your protocol to JGroups ?

    No. I guess that would be TOTAL_TOKEN, right?
  174. Java network performance[ Go to top ]

    ////
    > On that linked page, you don't specify the hardware that the tests were run on. The Sun-Blade 1000s are pretty high-end workstations, taking up to 4x US3 CPUs, at speeds from 750Mhz up to well over 1Ghz. Not publishing the hardware details (other than the memory size of 512MB) makes the results somewhat hard to gauge, particularly since jGroups seems to be very CPU intensive, and with multiple threads it should be able to take advantage of multiple CPUs.
    ////

    No, I used the plain SunBlade 1000 with just 1 CPU (running at 750MHz). I got better results on an average PC, although you can't really compare because of the different atchitectures.



    ///
    > If I read the report correctly, the 350 msgs/sec is actually 175 msgs/sec per sender, since you had two senders and you report the aggregate amount.
    ///

    Not really. I'm pumping 8000 msgs into the network, using 1 or 2 senders. So 1 sender doing 8000 or 2 senders doing 4000 each yields the same result.


    ///
    > Anyway, I wouldn't actually choose to point this all out, except you say that "A few hundred msgs/sec is actually pretty good" and go on to say in the report that "The tests show that JGroups performs consistently below Spread.... This is to be expected, as Spread is written in C". I disagree with both of these statements. The problem is definitely not the language of implementation;
    ///

    C is definitely faster than Java. But it is not only about that. With C you can use raw sockets for example, or hook into the network buffers to implement near-perfect flow control (like TCP does for point-to-point).


    > other pure Java implementations of reliable multicast

    which ones ?


    Bela
  175. Java network performance[ Go to top ]

    Cameron: .. other pure Java implementations of reliable multicast ..

    Bela: which ones ?

    I (and developers that I correspond with) have looked at packages such as SmartSockets (formerly Talarian, now Tibco), some of the JMS vendors that use multicast and of course our own TCMP implementation. For example, I did a quick test this morning with TCMP on three senders (each sender multithreaded, but each sender thread performing synchronous messaging) and the throughput was over four thousand messages per second. (APPLES AND ORANGES WARNING!!! It was definitely not the same test you were running and it was on different hardware, so it would be silly to compare the raw numbers ... I'm only trying to show ballpark numbers, particularly since I didn't have any dedicated receivers in my test -- the senders served double duty as receivers too.)

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Clustered JCache for Grid Computing!
  176. Generalized containers[ Go to top ]

    \Bela Ban\
    > JGroups (aka JavaGroups), the basis of JBoss clustering, does have a
    > distressing tendency to also eat critical exceptions

    Where ? I'd appreciate you telling me, so I can fix them.

    > and some alarming "ToDo" comments in the code.

    Where ? What is alarming ?
    \Bela Ban\

    [These were from me actually]

    I'll have to go through my notes and recheck some code to find all the relevant details. It's a rather time consuming process to not just find catch points, but to determine if they matter or not. At a very shallow end of looking at things, if you look through the code as a quick check of catch blocks, you can see many cases where a } catch {...} sets no indicators and takes no actions. I haven't re-checked the in-depth semantics of (it's been awhile and the code is complicated), but here are some examples with admittedly little analysis:

    in UDP.down():

            try {
                sendUdpMessage(msg);
            }
            catch(Exception e) {
                Trace.error("UDP.down()", "exception=" + e + ", msg=" + msg + ", mcast_addr=" + mcast_addr);
            }

    and UDP.sendMultipleUdpMessages():

            for(int i=0; i < dests.size(); i++) {
                dest=(Address)dests.elementAt(i);
                msg.setDest(dest);

                try {
                    sendUdpMessage(msg);
                }
                catch(Exception e) {
                    Trace.debug("UDP.sendMultipleUdpMessages()", "exception=" + e);
                }
            }

    ...plus things like:

        void sendUdpMessage(Message msg) throws Exception

    ...and:

            if(dest.getIpAddress().isMulticastAddress()) { // multicast message
                try {
                    mcast_sock.send(packet);
                }
                catch(Throwable e) {
                    Trace.error("UDP.sendUdpMessage()", "exception sending mcast message: " + e);
                }
            }
            else { // unicast message
                if(send_sock != null) {
                    try {
                        send_sock.send(packet);
                    }
                    catch(Throwable e) {
                        Trace.error("UDP.sendUdpMessage()", "exception sending ucast message: " + e);
                    }
                }
                else {
                    Trace.error("UDP.sendUdpMessage()", "(unicast) send_sock is null. Message is " + msg +
                                                        ", headers are " + msg.getHeaders());
                }
            }


    There could be a very good reason for this, but with no comments on why the exceptions are eaten with no action, or why Exception is being thrown out of a very core routine, it looks really, really worrisome.

    On ToDO's:

    TODO:
    -----
    resend() has to wait until it received all ACKs from all recipients (for all msgs), or until
    members were suspected. Thus we can ensure that all members received outstanding msgs before
    we switch to a new view. Otherwise, because the switch to a new view resets NAK and ACK msg
    transmission, slow members might never receive all outstanding messages.

    and:

       * todo: handle view changes (e.g. members {A,B,C}, blocked on C, and C crashes --> unblock

    and:

       TODO:
       - handle view changes and release locks held by crashed members


    The above sort of exception handling and comments does not give one the warm and fuzzies when thinking reliability and failure/recovery. They could be all red herrings, and I'd like to hear details on why these things are in the code, but it does look suspicious.

          -Mike
  177. Generalized containers[ Go to top ]

    I'll have to go through my notes and recheck some code to find all the relevant details. It's a rather time consuming process to not just find catch points, but to determine if they matter or not. At a very shallow end of looking at things, if you look through the code as a quick check of catch blocks, you can see many cases where a } catch {...} sets no indicators and takes no actions. I haven't re-checked the in-depth semantics of (it's been awhile and the code is complicated), but here are some examples with admittedly little analysis:

    >
    > in UDP.down():
    >
    > try {
    > sendUdpMessage(msg);
    > }
    > catch(Exception e) {
    > Trace.error("UDP.down()", "exception=" + e + ", msg=" + msg + ", mcast_addr=" + mcast_addr);
    > }
    >
    > and UDP.sendMultipleUdpMessages():
    >
    > for(int i=0; i < dests.size(); i++) {
    > dest=(Address)dests.elementAt(i);
    > msg.setDest(dest);
    >
    > try {
    > sendUdpMessage(msg);
    > }
    > catch(Exception e) {
    > Trace.debug("UDP.sendMultipleUdpMessages()", "exception=" + e);
    > }
    > }
    >
    > ...plus things like:
    >
    > void sendUdpMessage(Message msg) throws Exception
    >
    > ...and:
    >
    > if(dest.getIpAddress().isMulticastAddress()) { // multicast message
    > try {
    > mcast_sock.send(packet);
    > }
    > catch(Throwable e) {
    > Trace.error("UDP.sendUdpMessage()", "exception sending mcast message: " + e);
    > }
    > }
    > else { // unicast message
    > if(send_sock != null) {
    > try {
    > send_sock.send(packet);
    > }
    > catch(Throwable e) {
    > Trace.error("UDP.sendUdpMessage()", "exception sending ucast message: " + e);
    > }
    > }
    > else {
    > Trace.error("UDP.sendUdpMessage()", "(unicast) send_sock is null. Message is " + msg +
    > ", headers are " + msg.getHeaders());
    > }
    > }
    >
    >
    > There could be a very good reason for this, but with no comments on why the exceptions are eaten with no action, or why Exception is being thrown out of a very core routine, it looks really, really worrisome.




    What's bad about catching these exceptions ? This is not end-to-end, as with TCP sockets, we're just putting a packet on the wire. The upper protocols will retransmit if this fails.
    The only thing I could do here is to generate an event if for example the interface fails (NIC down), so that some code could failover to another NIC.
    Again, can't be done in Java. So what will happen in the above case is that the member will be shunned (expelled) and has to rejoin the group.

     


     
    > On ToDO's:
    >
    > TODO:
    > -----
    > resend() has to wait until it received all ACKs from all recipients (for all msgs), or until
    > members were suspected. Thus we can ensure that all members received outstanding msgs before
    > we switch to a new view. Otherwise, because the switch to a new view resets NAK and ACK msg
    > transmission, slow members might never receive all outstanding messages.



    This is under *DONE*. It was fixed Nov 30 1999 by me ! :-)





    > * todo: handle view changes (e.g. members {A,B,C}, blocked on C, and C crashes --> unblock
    >
    > and:
    >
    > TODO:
    > - handle view changes and release locks held by crashed members



    Yes - this is JBossCache. Come on, the JBossCache is beta, so what do you expect ? No items on the todo list ?

    Bela
  178. JGroups performance[ Go to top ]

    \Bela Ban\
    A few hundred msgs/sec is actually pretty good. Have a look at http://www.jgroups.org/javagroupsnew/docs/performance.html, where I compare performance for 10K messages in a 4-node cluster (2 senders). I get sustained throughput of 350 msgs/sec, that's 3.5MB/sec. That's slightly slower than Spread, but Spread is written in C !
    \Bela Ban\
     
    For my own purposes, I find most messages small, 1K or less, so I look at tests centering around that. From your tables, I see these which are most interesting:

      total_msgs: 1000/2000
      num_bursts: 100
      msgs_per_burst: 10
      msg_size (bytes): 500b
      sleep_time: 10ms

      Toolkit Senders Msgs/sec Throughput (KB/sec)
      JavaGroups 1 453 226
      JavaGroups 2 692 346

      total_msgs: 4000/8000
      num_bursts: 200
      msgs_per_burst: 20
      msg_size (bytes): 10000b
      sleep_time: 5ms

      Toolkit Senders Msgs/sec Throughput (KB/sec)
      JavaGroups 1 296 2968
      JavaGroups 2 451 4516

    I must admit, these tests strike me as rather odd. msgs_per_burst seems to vary from test to test, as does sleep_time. Please don't take things the wrong way, but it looks like you're skewing the tests for some reason. Why not show results with msgs_per_burst and sleep_time constant?

    Second, for the three tables (including the one I elided), messages_per_burst are 10, 5, and 20. What extremely low numbers!!!

    I've run my own tests with a proprietary messaging system using C and C++ and plain old TCP/IP (as implemented by a vendor of a RT messaging system). These tests are straight out w/out sleep time to test what happens when you overdrive the system.

    Tests are run on an HP-UX system on an L-class machine w/ 4 CPUs, around 800MHz, on 100MBit ethernet.

    In those tests, we routinely see messaging rates around 6000 msgs/second for 1K data payloads.

    For some of my own Java work, I achieve messaging rates around 2,800 messages/second in the same hardware setup using Java 1.4. I'm consistently just below half of the max rate of the C/C++ code.

    As I mentioned before, I find the burst-sizes rather bizarre, as well as the overall number of messages being sent. I can't conceive of a test involving between 1 and 8000 messages - the tests would be over in just several seconds including startup time!

    As a sort of sanity check, check JMS vendors' messaging rates in their benchmarks when disk-forcing is turned off. You'll see similar results to mine (mine tend to be a little better only because my cited example isn't JMS, but a home-grown protocol).

    I have noticed that JGroups seems to rely on Java object serialization quite a bit, and this could be a source of _alot_ of performance angst. My own tests with a protocol that uses all-Externalizable objects, tops out around 800 messages/second - quite comparable to your results. In general serilization should be avoided like the plauge when doing lowest-level messaging code - it eats CPU like there's no tomorrow.

         -Mike
  179. JGroups performance[ Go to top ]

    In general serilization should be avoided like the plauge when doing lowest-level messaging code - it eats CPU like there's no tomorrow.

    >
    > -Mike

    More than that there is a synchronization bottleneck in ObjectStreamClass. In JDK 1.5 I hope they move to ConcurrentHashMaps for the global caches they have in ObjectStreamClass.

    Bill
  180. JGroups performance[ Go to top ]

    As I mentioned before, I find the burst-sizes rather bizarre, as well as the overall number of messages being sent. I can't conceive of a test involving between 1 and 8000 messages - the tests would be over in just several seconds including startup time!


    I just went back to look at Bela's stats. He does seem to be running a test with a total of only a few thousand messages. How can you measure anything like that? It's like weighing the captain on the ship and subtracting the weight of the ship.

    Whatever, I guess we are way off topic and this horse is dead ..
  181. JGroups performance[ Go to top ]

    ///
    > I must admit, these tests strike me as rather odd. msgs_per_burst seems to vary from test to test, as does sleep_time. Please don't take things the wrong way, but it looks like you're skewing the tests for some reason. Why not show results with msgs_per_burst and sleep_time constant?
    ///


    No, no problem. As I mentioned, the tests were written by ADAPT, not by me. (They wrote a paper claiming that JBossCluster falls apart under heavy load.) I simply re-ran their tests for (a) JGroups and (b) Spread, and I use the flow-control config with JGroups, rather than the default one.

    I didn't change their tests at all to compare the results.

    Yes, burst rate and sleep times are weird, but I wanted to run exactly the same tests

     
    ///
    > For some of my own Java work, I achieve messaging rates around 2,800 messages/second in the same hardware setup using Java 1.4. I'm consistently just below half of the max rate of the C/C++ code.
    ///

    Can you send me those tests ? I'd like to run JGroups against them.
    Those numbers are excellent: a msg takes ~ 300 microsecs. Do you run this on TCP or UDP ? Do you include retransmission ? What about fragmentation ?

      
    ///
    > As I mentioned before, I find the burst-sizes rather bizarre, as well as the overall number of messages being sent. I can't conceive of a test involving between 1 and 8000 messages - the tests would be over in just several seconds including startup time!
    ///

    Agree. I always notice a ramp up time, after which the messages/sec rate stabilizes. I get much better results sending 100000 msgs than sending 8000 msgs.




    > I have noticed that JGroups seems to rely on Java object serialization quite a
    > bit, and this could be a source of _alot_ of performance angst.

    We tried to get rid of serialization as well as possible. What I put on the wire is a Message, consisting of
    - src, dest: InetAddress plus port (the equivalent of InetSocketAddress in 1.4)
    - payload: byte[] buffer
    - headers: HashMap of keys (String) and values (Header subclasses, Externalizable)

    Each protocol adds/remove entries from the headers hashmap. Here I may go with ByteBuffers in the future. Also, InetAddress in 1.3 doesn't have a good conversion from InetAddress --> byte[] --> InetAddress, so we wrote our own thing. This is superfluous in 1.4.

    Bela
  182. Generalized containers[ Go to top ]

    There are also many missing extras that just about every other implementation

    > I know of has, such as dead message queues and the like

    ??

    My JBoss has DLQ right here...

    And then you call the JBG guys incompetent and liars. What a fool YOU are.

    /T
  183. Generalized containers[ Go to top ]

    I've read just about every line of their JMS system


    I guess you haven't. So who's the liar now? No DLQ you say? You're nothing but a buffoon.

    /T
  184. Generalized containers[ Go to top ]

    \Thomas Mattson\
    I guess you haven't. So who's the liar now? No DLQ you say? You're nothing but a buffoon.
    \Thomas Mattson\

    D'oh! My head got mixed up on that one, and you are absolutely correct - there is a DLQ there, and there has been since 2001. That's what I get for only consulting my notes for only 80% of the post and trying to round out the remainder from memory. For the record, I've checked the remaining items I listed and they do indeed apply.

    But I guess in your eyes by making an honest mistake I'm a buffoon. So be it.

       -Mike
  185. JBoss Clustering[ Go to top ]

    <Clustering is a very murky subject. JBoss people claim fabulous things for their clustering code. Funny thing is, try to find someone who actually uses JBoss clustering in production. I've only found a handful, and the bulk of them are doing fairly vanilla HTTP session replication. Look into the underlying code, and you see alot of information coming in which isn't checked for validity, a number of exceptions which are just caught and printed out and never dealt with, and a mountain of unanswered cries for help for problems on the cluster forum.
    <
    I am astonished that you would throw this criticism in with very little to back it up. Your reviews of the source code are all fine and well, but without having the benefit of contrast with BEA's or X's source code, what's the point ?

    Moreover, the fact that you personally have not found anyone implementing the [non-HTTP] clustering is a weak indication of the product's ability to function as advertised. I spent belts of time at an investment bank watching WLM consistently and miserably fail, so I am delighted with the elegance and functionality of JBoss clustering.

    While I cannot claim to have tested or implemented all of the clustering features, the combination of auto-discovery and EJB round-robined remote invocation with seamless failover has proved to be flawless, and very simple to implement. Add to that the ability to add and remove servers from the cluster to allow for load management and rolling upgrades, I think you are being unduly critical in this respect.
  186. Generalized containers[ Go to top ]

    Gugliemo: If you can give a good summary of what you know about the real JBoss, I'd love to hear to it. My impression so far is that
    a) It's basically compliant with J2EE (99% say)
    b) It scales about as well as Tomcat.


    JBoss includes Tomcat (or Jetty) to handle that part of the J2EE spec, so it scales almost exactly as well as Tomcat ;-). The "JBoss part" is the JMS, EJB, J2CA, etc.

    As far as compliance, it's really pretty good for 99% of what most people do with J2EE. It's easier to develop with than most app servers -- seriously, it is. And I'd be fairly confident in saying that it will become certified soon, since the big money clock ($500k for one year for certification?) is ticking for JBG. (My guess is that one reason that 4.0 is late is that 3.x is being made certifiable. It's a reasonable trade-off.)

    As far as scalability, it's mostly up to the architecture of the application. I know of some pretty high scale applications running on JBoss, but -- not surprisingly -- the reason I happen to know about them is that they are clustered using Coherence.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Clustered JCache for Grid Computing!
  187. Generalized containers[ Go to top ]

    JBoss includes Tomcat (or Jetty) to handle that part of the J2EE spec, so it scales almost exactly as well as Tomcat ;-). The "JBoss part" is the JMS, EJB, J2CA, etc.


    I meant the parts other than the servlet engine.

    > As far as compliance, it's really pretty good for 99% of what most people do with J2EE. It's easier to develop with than most app servers -- seriously, it is.

    Okay

    > As far as scalability, it's mostly up to the architecture of the application. I know of some pretty high scale applications running on JBoss, but -- not surprisingly -- the reason I happen to know about them is that they are clustered using Coherence.

    I am inclined to believe you.
  188. Generalized containers[ Go to top ]

    I just don't see that. If you want to consult, then do it, but why consult on a single product? Maybe I should see JBoss taking over the world,

    >

    FOR THE 100TH TIME We don't consult. We do ZERO implementation for example. We will do DEVELOPERS SUPPORT in the "help me with my JBoss architecture/problem" kind of thing. We NEVER take projects we refer them to our JASP partners around the world. That is the kind of work they do. We do 3rd line JBoss support for our JASP partners.

    We mostly sell PRODUCTION SUPPORT and maintenance on JBoss as well as training on internals. We do that on JBoss, we are the most credible to do it on JBoss.

    >  
    > > JBoss has a reputation for cutting edge techno extremists.
    >
    > Mostly you have the reputation of a guy who thinks he knows it all and who thinks everybody else knows nothing.
    >

    he he

    >
     I doubt he insults people like you do. And as far as I can see he maintains his projects without bragging about it to everyone.
    >

    right... go read his posts on guns and stuff.

    marcf
  189. Huh?[ Go to top ]

    Not all companies use opensource technologies. You are tying yourself to a "product", not a technology. Hibernate is a product. JDO is a spec.


    I just want to add that "tying yourself to a product" is only bad because there is a commercial company behind those products: once you are locked in, the cost can go up. On the other hand, if the price is always zero (as is the case with Apache, Linux, Hibernate etc.) then being tied to the product is not a bad thing.
  190. Roadmap to the Wrong Place[ Go to top ]

    IMHO it's useless to write an open-source implementation of J2EE. The purpose of specs, like Java, J2EE etc. is to manage risk. On the other hand, the purpose of open-source is also to manage risk. If I use a spec-compliant product, I can always replace it with another one. If I use an open-source product, the likely that I'll need to replace it is almost nil.

    >
    > So, specs and open-source are really an either/or problem. You can have both (JBoss), but what's the point? Specs can be good, but usually they are not that great, and often they are REALLY bad (and politically motivated.)
    >

    useless to write an open source impl? non sensical logic, right analysis, wrong conclusion.

    You are correct that open source has the potential to be a defacto standard in the presence of paper standards... IT IS FREE AND OPEN. Once you are there the chance you will start paying more for an inferior server is almost nil :)

    The real problem is the presence of an implementation (.NET) by a monopolist in front of a spec (J2EE). Monopolies aren't bad, in fact I argued in the past that Free Software is monopolistic (by the community). But in reaching that monopoly impl >> paper alway. Implementation beats paper.

    So we decided to lead with implementation and let the spec catch up. We paid for the freaking J2EE certification but we push all the pojo middleware stuff as the right form for modern middleware. For example I am really pleased with the direction EJB 3.0 is taking, it is looking more and more like JBoss 3.x :)

    So an implementation will always win, we hope it to be JBoss and are serious about it. We want to be the monopoly :) you know it will be a kinder one than if non-techies own it.

    > Just look at how well Hibernate is doing.

    Which is why we decided to bring hibernate on as a JBoss project. Gavin and Hibernate today are professional open source and you are correct in identifying the trend of "non-standards driven java" but I believe another post delves deeper in the standards adoption by users, I will respond to it.

    marcf
  191. Stand alone components?[ Go to top ]

    I hope they will untangle more components. JCA is still only available in EJB servers, as far as I can tell. Standalone JMX could be used as the basis of a management framework. Components such as Hibernate, JBossCache, JGroups are useful without the app server JBoss should appreciate the idea that they have components that are, or can be, more popular than the app server and make the most of it.

    Hibernate uses CGLib as do several AOP frameworks because it doesn't penalize performance. Can we expect JBoss to follow suit to avoid a bytecode weaving clash between JBoss and Hibernate?

    Gary
  192. JBG,

    Thanks for letting us know what is coming up and what new things we can look forward to. I think this document also lets people know where they might be able to help, if they want to.

    Steve
  193. JBG,

    >
    > Thanks for letting us know what is coming up and what new things we can look forward to. I think this document also lets people know where they might be able to help, if they want to.
    >
    > Steve

    This is exactly what it is there for. We're trying to outline fine-grained tasks so that it is easier to contribute and for people to more easily discover how they can contribute. Thanks to Ivelin Ivanov for donating some of his free time and energy in putting this document together.


    Bill
  194. Direction for JBossGroup?[ Go to top ]

    From all the discussions that I have had with Marc over the years, I have concluded that JBossGroup is more important than the code itself. In fact, the code mainly exists to sustain JBossGroup.

    Given this realization, perhaps JBossGroup should expand its horizons and offer support not only for JBoss but also for other app servers like BEA and WebSphere (in fact this would make the vendors really mad because it would directly subtract indispensable support dollars.) JBoss should be its recommended platform, but not its *only* platform.

    It just doesn't make good business sense. Since it's clear that BEA and WebSphere are the default choice for J2EE, if you limit yourself to JBoss you are going to miss a lot of business, with the potential risk of eventually disappearing altogether.
  195. Direction for JBossGroup?[ Go to top ]

    Guglielmo lichtner,

    > From all the discussions that I have had with Marc over the years,

    ...

    Really? when? I don't know you.... at all.

    I never met you and I don't recall ever exchanging emails with you... to me you are a TSS shadow-alias, so maybe if you would give us another name you go by I could validate that statement :)

    >
    I have concluded that JBossGroup is more important than the code itself. In fact, the code mainly exists to sustain JBossGroup.
    >

    not more important but EQUAL.

    JBoss inc == JBoss. JBoss == JBoss inc. Get it?

    There is no distinction in my mind between the product and the company. The two are one and the same, symbiotic as bill says. The company is here to fund the development of the code. The quality of the product means we have a business.

    That being said the time we break that equation is the other way around. JBoss is funding Tomcat for example. Remy is on board from France and leads TC4 and TC5. Sure Tomcat is important to JBoss inc' business as we provide support for it but it is a two way street. Putting one or the other above isn't true.
     
    > Given this realization, perhaps JBossGroup should expand its horizons and offer support not only for JBoss but also for other app servers like BEA and WebSphere (in fact this would make the vendors really mad because it would directly subtract indispensable support dollars.) JBoss should be its recommended platform, but not its *only* platform.
    >

    cute, however JBoss== JBoss inc. So JBoss inc can't do support of BEA (god forbid!) and we wouldn't be credible.

    marcf
  196. Direction for JBossGroup?[ Go to top ]

    Guglielmo lichtner,

    >
    > > From all the discussions that I have had with Marc over the years,
    > ...
    >
    > Really? when? I don't know you.... at all.
    >
    > I never met you and I don't recall ever exchanging emails with you... to me you
    > are a TSS shadow-alias, so maybe if you would give us another name you go by I
    > could validate that statement :)

    Let me remind you. On 2001-11-30 you sent me (you do remember me, right?) an email with subject "gugielmo" with body as follows:
    "
    IS REALLY CEDRIC DID YOU SEE THE LAST ABOUT "WORLD EXPERTS"?

    marcf
    "

    /R
  197. Direction for JBossGroup?[ Go to top ]

    Let me remind you. On 2001-11-30 you sent me (you do remember me, right?) an email with subject "gugielmo" with body as follows:

    > "
    > IS REALLY CEDRIC DID YOU SEE THE LAST ABOUT "WORLD EXPERTS"?

    I am flattered that somebody might think that I am a "front" for such an authoritative personality. I do work at BEA. However, I really *am* a real person and I have never met Cedric, and unfortunately I don't think BEA would ever offer me a job :(
  198. Direction for JBossGroup?[ Go to top ]

    Really? when? I don't know you.... at all.


    Perhaps I was inaccurate. I certainly did not mean to imply that you are a friend of mine :-)

    > JBoss inc == JBoss. JBoss == JBoss inc. Get it?

    This is not a good thing. I think people understand why and therefore I don't need to explain.

    > cute, however JBoss== JBoss inc. So JBoss inc can't do support of BEA (god forbid!) and we wouldn't be credible.

    I don't see why not. You will be as credible as you are now.