Combining the Servlet API and NIO tutorial & comparison

Discussions

News: Combining the Servlet API and NIO tutorial & comparison

  1. There have been numerous discussions on TSS about whether Servlets could be retrofitted to work with NIO. Taylor Cowan did a prototype of a Servlet container with NIO and ran some tests. The results were that HTTP 1.0
    open socket/close socket clients will not benefit much from NIO, however, SOAP clients or HTTP 1.1 clients that hold onto sockets longer will see some improvements from a server that uses NIO selectors.

    Read Taylor Cowan's The Servlet API and NIO: Together at last on developWorks.

    Threaded Messages (22)

  2. At least Reattore shows some really interesting performance gains by using NIO. Reattore was written about a year ago by Michael Hope.

    http://reattore.sourceforge.net/
  3. For a nice discussion of using the NIO APIs to build network software, see this article from Java Developer's Journal: http://sys-con.com/story/print.cfm?storyid=36986
  4. There is an interesting discussion on Event based programming vs. Threaded programming at
    http://lambda.weblogs.com/discuss/msgReader$10796?mode=topic&y=2004&m=2&d=4.
  5. Recommended reading[ Go to top ]

    Hi!

    You might also want to look at the ACE server framework by Douglas Schmidt. He made all aspects of server request processing pluggable via the Strategy pattern and did numerous research papers on the characteristics and benefits of each combination.
  6. More on ACE[ Go to top ]

    Schmidt does really great work on should be
    referenced more. His site is at
    http://www.cs.wustl.edu/~schmidt/ACE.html.
  7. More on ACE[ Go to top ]

    I worked quite a bit with ACE (and TAO) in C++ when I worked with telcos. It's an astoundingly good library. ACE is based on generalized patterns described in Pattern Oriented Software Architecture Vol. 2. For network programming, I found C++ with ACE to be more productive than Java; I still feel that's true, though Java has come a long way.

    Two other good books that cover ACE are Schmidt's C++ Network Programming, vols. 1 and 2. I haven't read Steve Huston's book on ACE, though my company at the time used his services and he clearly knew the framework well.

    Great stuff to know if you're working on middleware or servers more generally.
  8. Recommended reading[ Go to top ]

    Hi!

    You might also want to look at the ACE server framework by Douglas Schmidt. He made all aspects of server request processing pluggable via the Strategy pattern and did numerous research papers on the characteristics and benefits of each combination.
    http://www.cs.wustl.edu/~schmidt/patterns-ace.html
  9. Good article. We at Borland went down similar path to provide a NIO connector to our Tomcat based Web Container in Borland Enterprise Server (BES) 6.0. What we saw was at lighter loads, NIO actually decreased the performance. However for clients in excess of 200, NIO based connector showed real promise. For a 3 tiered server (servlet, EJB, Database), it is very important to limit the number of server threads. NIO makes it possible to limit the server threads without actualy limiting the number of clients.

    The JDK bug 4729342 stopped us from reaching very high loads. From the code sample posted in this article, it appears that the author also experienced this bug.

    For some reasons, I could not download the code from IBM web site. Download always timed out on me.

    NIO connector for BES 60 is available for tech preview.
  10. This is wrong.[ Go to top ]

    This overly simplified example (just HTTP gets) doesn't tell you anything IMHO. The reality is different when you have a conversational state with a client. A more complex server that has to deal with both incoming and outgoing data (and some of this data are serialized Java objects) the managing of the queue and 'homemade' context switches gets harder and more error prone.

    One thread/client is a natural way to deal with that kind of problem. Almost the whole server logic can be coded with just one client in mind.

    Why not hide the implementation in threading layer?

    take a look at this: http://www.sics.se/~joe/apachevsyaws.html

    Yaws is a HTTP high perfomance 1.1 webserver where one Erlang light weight process is used to handle each client. Just to discard the though that some kind of 'low number of threads kludge' is needed to maintain throughput, Yaws maintains it's pace even with *80k* of simultaneos connections EACH HAVING A SEPARATE PROCESS. The selector/queue mechanism described in the article seems to choke on +4k connections...

    Why should this plumbing be invented for each server again and again? Let the runtime handle threading issues so that coders have more time to focus on application level problems. Erlang does not make use of the underlying OS's threads and processes for managing its own process pool and thus does not suffer from these limitations.

    (NOTE: some text snippets where copypasted from Yaws website)
  11. This is wrong.[ Go to top ]

    agree with Nipsu. there is no intrinsic reason why one programming model is slower than another one. if anything needs to be fixed, fix the thread implementation, don't let people use a equivalent but worse programming model.
  12. JRockit ?[ Go to top ]

    Well, JRockit's thin threads came to solve this but they're not the default: http://edocs.bea.com/wljrockit/docs81/userguide/threads.html#1008124. Looks like they've been experimental for years. Is anyone using them ?
  13. SEDA[ Go to top ]

    Matt Welsh has done some very interesting research on that already back in 2002
    http://www.eecs.harvard.edu/~mdw/proj/seda/
  14. SEDA[ Go to top ]

    Matt Welsh has done some very interesting research on that already back in 2002


    The last update seems to be from 20020712. Does anyone have any experiences with Seda or Sandstorm on how they behave with latest sun NIO code? Is the project abandoned and if so, why?
  15. SEDA[ Go to top ]

    Currently i'm trying to create a seda-based servlet container, but i've just started so I can't present any results yet. The project seems to be abandoned, that's right. Remember that Sandstorm was(is) only a prototype and probably mr.Welsh found something more interesting to do (it seems he created all the code alone). Nethertheless it works. Also with nio, but it was not tested with it. However some tests (perl scripts) come with Sandstarom distribution. Sandstorm contains some useful tools, such as profiler, socket lib, simple http lib, async file i/o (simulated by using threads). There is also slow async SSL implementation and a http server called Haboob (Habob?) (separate project) which was compared against apache 1.3 and Flash. The results of this comparison show that Seda + async io (using NBIO) is unbeatable under heavy load. In fact dealing with heavy load is the main purpose of Seda. Async io gives the ability to serve thousands of clients at a time (imagine the thread sync overhead in a standard multi-threaded server). The programming model hides threading issues from the user, which may seem fine, but is not flexible enough (you may only create a single-threaded stage or any-threaded stage (number of threads is controlled by the framework at runtime to maintain target response time)). The interface EventHandlerIF (which is the main entry point of the stage)is quite similar to servlet interface and maintaining a state is not supported by the framework (which is actualy good, because snadstorm is quite lightweight and it was not intended to become a small ejb container).
  16. Tried this with Jetty[ Go to top ]

    I have blogged about trying this appraoch on Jetty at

     http://www.mortbay.com/MB/log/gregw/

    The short summary is that while more promising than other NIO
    attempts, Taylors technique (or my implementation of it) has
    not produced an easy win for Jetty. Thus some more thought/work/ideas may
    be needed before this approach can be applied to a real servlet
    container.
  17. Tried this with Jetty[ Go to top ]

    Greg, some questions and observations on the code:

       - What OS and JVM was this tested on? Various Unixen and Windows behave quite differently with NIO. If you were testing under Windows, I wouldn't use that as a gauge as to what you might see under Linux, Solaris, HP-UX or others.

       - The write() code in ByteBufferOutputStream flushes the NIO buffer on every single write. You gotta know that that's not going to perform well.

       - What was the CPU utilization of the machine under varying loads? What were the I/O rates? How do these compare with non-NIO?

       - ByteBufferInputStream seems to use synchronized methods everywhere - why? You'd expect only one reader at a time, wouldn't you?

       - doRead() does a one-shot attempt at reading. In my experience, sockets will often return with a small number of bytes available (like most often 1), and a second read attempt will return most/all of the incoming data.

       - The info you've posted on the cost selection set management doesn't jibe at all with my numbers. Same with the cost of changing blocking modes. Again, perhaps this is a Windows thing and not necessarily an overall NIO thing?

    Overall, it appears to me that you're comparing optimized non-NIO Jetty code against some NIO code which was hastily put together and not tuned in any meaningful way. Additionally, I don't see any indication that tests were performed on multiple JVMs or OS platforms.

    Don't you think you're being a bit hasty jumping to the grand conclusions you make in your blog? The tone of your blog entry conclusions is that you've somehow conclusively proven that NIO performance sucks in the latency department, but you're basing those conclusions on some pretty shaky grounds.

    You say "the latency of the server under load has doubled". Well - find out why! Is it really NIO, or is it the OS, or is it the implementation in your code?

         -Mike
  18. Tried this with Jetty[ Go to top ]

    As a quick followup....doing the accept() calls in the same thread as the read() calls is going to slow you down even more if you typically have frequent connection attempts by clients.

        -Mike
  19. Simple[ Go to top ]

    Has anybody tried to use SimpleWeb?
    http://simpleweb.sourceforge.net/

    I like simple ideas that come in small packages and this one seems to deliver too. =)

    No bloat there. Just webstuff(TM) that works!
  20. Experience with Simple[ Go to top ]

    I have been using Simple for quite a while now and it really does perform much better that most of the Servlet engines out there such as Tomcat. We use it for our corporate Intranet site which, in the past, tended to get overloaded frequently....Works well with Velocity.
  21. Performance of Simple[ Go to top ]

    In general Simple tends to perform about twice as fast as Tomcat under stress. Also, as mentioned in the above thread, Simple tends to come into its own when the load increases and the parallelism required is high.

    Another possible benifit of NIO is that is perhaps not mentioned is that not as many requests would be dropped. Benchmarks that I have seen on Servlet engines such as Resin, Jetty, and Tomcat tend to mention response rate but fail to provide any data regarding failures.

    I recently performed a performance comparison netween Tomcat and Simple, and noticed that Tomcat drops an enormous quantity of requests in comparison, perhaps NIO implementations can address this?
  22. Performance of Simple[ Go to top ]

    One more thing.. does Simple support SSL connections? There were no indications about that in simpleweb.sf.net?
  23. SSL Support[ Go to top ]

    Simple does/can support SSL. The JSSE libraries can be used to create an SSLServerSocket which can be passed to a Connection. The JSSE examples provide an implementation that is very similar to what can be done to suport SSL/TLS with Simple.