Netty2 1.8 Released: Event-driven NIO framework

Discussions

News: Netty2 1.8 Released: Event-driven NIO framework

  1. Netty 2 provides an easy event-based API (like Swing) to develop high-performance, maintainable TCP/IP server/client application. Netty provides many essential features such as readiness selection, thread pooling, write buffer DoS prevention, and buffer reuse which are required to build high- performance and capacity network applications.

    In this release, users can catch internal or uncaught exceptions setting his or her custom ExceptionMonitor implementation and can set timeout for Session.write() to avoid deadlocks when maxQueuedWriteCount is set.

    Netty 2's interesting features are: (Please request me if there should be more)

        * Event-based API: Netty handles all low level I/O call
        * Separation of message format and workflow
        * Protocol messages are represented as classes: Protocol implementation in object-oriented way!
              o Polymorphism enables 'pluggable protocol'.
              o Inheritance enables 'message part hierarchy'.
        * Java NIO: Better scalability
        * Built-in thread pool:
              o I/O and event processing threads are separated.
              o All thread priorities are configurable.
        * Direct byte buffer pooling: No direct buffer allocation overhead
        * Customizable event dispatcher: Flexible thread scheduling
        * 'Write buffer full' prevention:
              o Detects too slow clients
              o Throttles outgoing messages to avoid resource shortage
        * JMX-compliant: Thread pools and common configuration are MBean.
        * Supports Apache Jakarta Commons Logging.

    Netty 2 is good for: (Please report me if there are more)

        * implementing both legacy and non-legacy protocols fast but in high-quality
        * implementing pluggable protocols (different message formats with one workflow)
        * creating test cases both for message formats and protocol workflow

    Netty 2 home

    Threaded Messages (22)

  2. Great job, Trustin! Very handy framework.

    --
    Dimitri
  3. Thank you Dimitri! Are you using Netty2? Please let me know your use cases if you can tell me. :)

    Cheers,
    Trustin Lee
  4. UDP support?[ Go to top ]

    Hi,

    is there planned support for UDP protocol? I read about that in FAQ but there is no date's when it will be available ...

    greetings

    Michal
  5. Now that the documentation is getting better, Netty is becoming interesting.

    Good work!
  6. Great job! Easy to use and very well designed.
  7. According to the forum, Trustin, you say that Netty2 is going to be deprecated???
  8. It sounds like, from the forum, that Netty is sort of end of life'ed, and that the Apache SEDA project is going to take its place. I didn't actually know that Apache was working on a SEDA project, but also according to the forum, it still has quite a ways to go. So I guess there is a gap in time while this is worked out, though he did mention that there would likely be a compatibility layer to migrate projects over.

    I'm curious what this change in model.. to the SEDA model.. will do to applications that are expecting to be thread-based. Many JSP/Servlet apps, and their accompanying template/presentation systems are expecting to have a single thread for the duration of the request. I have no idea, but would expect that many persistence libraries also have the same sort of expectation.

    If we decouple one portion of the system from a threaded model and move to a SEDA/Event model, what does this mean for the rest of the processing system? What would happen to Tomcat, for example, if we used a Netty/SEDA layer for network processing? I would think even if the request processing was thread-bound, we could still get some scalability enhancements..
  9. It sounds like, from the forum, that Netty is sort of end of life'ed, and that the Apache SEDA project is going to take its place. I didn't actually know that Apache was working on a SEDA project, but also according to the forum, it still has quite a ways to go. So I guess there is a gap in time while this is worked out, though he did mention that there would likely be a compatibility layer to migrate projects over.

    Yes, I'm releasing Netty2 only applying essential bug fixes and improvement for now. That's why I'm not supporting UDP for Netty2. Please use Netty2 and report any bugs of it, then new maintenance version of Netty2 will be released and Apache SEDA will get better from it. :)
    I'm curious what this change in model.. to the SEDA model.. will do to applications that are expecting to be thread-based. Many JSP/Servlet apps, and their accompanying template/presentation systems are expecting to have a single thread for the duration of the request. I have no idea, but would expect that many persistence libraries also have the same sort of expectation.

    In Netty2, I/O and message encode/decode phase were merged into one thread pool, but this will be separated to single (or two) read/write threads and a codec thread pool to enhance performance. There will be a big API change, because SEDA model divides network application process into Accept, Input, Decode, Process, Encode, Output phases, but Netty2 does in somewhat ambiguous way.

    Anyway I believe creating a compatibility layer should be done by keeping Session and Message interface still usable. :)
    If we decouple one portion of the system from a threaded model and move to a SEDA/Event model, what does this mean for the rest of the processing system? What would happen to Tomcat, for example, if we used a Netty/SEDA layer for network processing? I would think even if the request processing was thread-bound, we could still get some scalability enhancements..

    Yes, possibly. But SEDA could be the base framework of Tomcat, James, or even AXIS actually. Eve (an LDAP server) is currently based on SEDA.
  10. I'm curious what this change in model.. to the SEDA model.. will do to applications that are expecting to be thread-based. Many JSP/Servlet apps, and their accompanying template/presentation systems are expecting to have a single thread for the duration of the request. I have no idea, but would expect that many persistence libraries also have the same sort of expectation.

    I think I can say something obout SEDA, Netty and Tomomcat.
    But first about servlet/jsp api: it's 100% blocking api and you CANNOT change thread during servlet execution (i mean the same request ). Why? because the api doesn't allow this. Simple and cruel. There are however two things one could do in separate stages, these are:
    1. Parsing http header
    2. Finishing the response (flushing buffers, etc.)

    Getting back to Tamcat, SEDA and Netty. I TRIED to make Tomcat connector using seda (original impl, not talking about new apache project) and netty. And it was almost as fast as original coyote connector (= It (I) failed).
    And after 6 months of work my experience is:
    1. Original SEDA is a crap
    2. Original SEDA async socket i a crap^2 (that's why i decided to use netty)
    3. You have to do nonblocking http parsing (the one easy thing)
    4. Even if you do nb parsing, you still have a big problem with the stage which is executing servlets. Because servlets sometimes call blocking api (e.g. reading file, doing JDBC an so on) you have to use a thread pool controller to add threads to the pool as some of them are blocked. But doing so you are misusing SEDA because it is meant to run only a few threads. So you end up with a stage having almost 1 thread per connection (or a huge queue). You can limit the number of threads, but then response times tend to grow (sometimes exponentially).
    5. Even if you make it fast, you cannot reduce memory consumption. Using nio, you have to allocate a pool of direct buffers (funny thing - direct buffers are garbage collected very, very slowly ). Because SEDA can accept more incoming connections (some are waiting in queues) you need to allocate more memory. Remember that you have to provide 2 streams for each servlet request, and each such stream must use a buffer.
    6. Tomcat 5 is not well designed inside (in fact it's quite messy ). If i were trying do make nio enabled servlet container, i'd choose jetty.


    So tu sum up. Nonblocking io is good for simple things. But not for servlet api. It was designed to be used by servers using thread per request model.

    And a note about SEDA. The idea looks nice and there is only one constraint. You cannot block in stages. Now try to imagine as much programs that don't need to block as you can. Lest's say in 5 minutes.
    ...
    And ? how many ?

    P.S. Sorry for my english ;)
  11. Netty2 1.8 Released: Event-driven NIO framework[ Go to top ]

    it's funny how often this topic comes up. When Matt Welsh first published his results I spent a lot of time reading his papers, SEDA and Haboob.

    My own conclusion have 2-3 years of exploration led me to the conclusion that retrofitting the Servlet API to fit an event driven approach like SEDA is hard enough as to be pointless. Not only would it introduce more complexity to the servlet API, but in the end you'd probably have a slower servlet container. though I'd love to be proven wrong by the servlet spec team and have them find an easy way to make SEDA fit.

    I don't agree that SEDA is crap :) It is hard as hell to understand and it's not something that comes easily. Especially if you're like me and most of the day-to-day work is servlets. Breaking a webapp into separate stages takes mental gymnastics (probably just me). Let's face it, in most cases, using a nice caching mechanism makes it easy to scale out and requires less work for developers. I'm totally generalizing here, but I believe most developers just aren't in the practice of thinking in SEDA.
  12. We've talked about this at length...[ Go to top ]

    ... on the tomcat-dev list, with a couple of developers going as far as creating some prototypes for parts of Tomcat based on java.nio and/or SEDA. The resulting performance wasn't as good as the current implementation, and we had consensus that it'd be extremely difficult (if possible at all) to maintain conformance to the Servlet Spec, which as others have pointed out is really is difficult from a threading perspective.

    As to Lukasz Kubica's claim that "Tomcat 5 is not well designed inside (in fact it's quite messy ). If i were trying do make nio enabled servlet container, i'd choose jetty." -- please. Have fun ;)
  13. We've talked about this at length...[ Go to top ]

    If your servlets are for simple request-response type
    apps then SEDA probably isn't going to buy you much. But if you use servlets as a generic API hiding complex behaviours and you have high loads, then a SEDA architecture can be a big win.
  14. We've talked about this at length...[ Go to top ]

    If your servlets are for simple request-response typeapps then SEDA probably isn't going to buy you much. But if you use servlets as a generic API hiding complex behaviours and you have high loads, then a SEDA architecture can be a big win.

    Please explain why ..

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Shared Memories for J2EE Clusters
  15. We've talked about this at length...[ Go to top ]

    Please explain why ..

    If all you do is get a record and format a page then what are your possibilities to speed this up? There just aren't a lot so SEDA wouldn't help much.

    Now if you have quite a bit of orchestration then you have some room for architectural improvements.

    And load is really the key. It's under load where architecture can help meet concerns for QOS, latency, priority, throughput, connection acceptance, shedding work, etc.

    If you don't have such concerns and/or your loads are low, then pretty much anything works well enough.
  16. We've talked about this at length...[ Go to top ]

    If all you do is get a record and format a page then what are your possibilities to speed this up? There just aren't a lot so SEDA wouldn't help much.

    OK, so you are saying that SEDA doesn't speed up Servlet-style processing? Or it doesn't speed up the simple web-style Servlet processing?
    Now if you have quite a bit of orchestration then you have some room for architectural improvements. And load is really the key. It's under load where architecture can help meet concerns for QOS, latency, priority, throughput, connection acceptance, shedding work, etc.If you don't have such concerns and/or your loads are low, then pretty much anything works well enough.

    The problem I'm having is getting the specifics from what you are saying. We can assume "high load" if that helps to explain things. I see you mention "connection acceptance," so I assume that SEDA handles more "socket accepts" per second on a given port, for example? If so, can you explain why?

    You also mention:

    - QOS - What additional QOS does it provide over and beyond TCP/IP?

    - latency - This one seems backwards, as high-throughput systems are not designed to minimize latency; could you please explain further.

    - priority - Please explain. Are certain connection prioritized? Or threads?

    - throughput - this one I think I can guess at, since NIO allows you to handle lots of socket I/O throughput using relatively few resources (specifically threads,) which means that you can theoretically achieve much better throughput under certain circumstances. So is the throughput improvement related to the use of NIO, or is there more to it than that?

    - shedding work - I don't know what you mean here at all; please explain.

    I haven't had a chance to go through SEDA in detail; pardon my ignorance. I've studied NIO quite a bit, as well as EmberIO, so I have some points of reference if that helps to explain. I'd just like to get a clearer idea of what is actually there, and why it accomplishes all of these things.

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Shared Memories for J2EE Clusters
  17. We've talked about this at length...[ Go to top ]

    I don't wish to waste everyones time, so here some of papers that may explain some of the issues i mentioned. Perhaps i have confused things because i am not talking only about NIO, but of application architectures in general.

    * http://www.possibility.com/epowiki/Wiki.jsp?page=ArchitectureDiscussion
    * http://www.possibility.com/epowiki/Wiki.jsp?page=AppBackplane
    * http://www.possibility.com/epowiki/Wiki.jsp?page=Scalability
    * http://www.possibility.com/epowiki/Wiki.jsp?page=HandlingInfiniteWorkLoads
  18. Another point of view[ Go to top ]

    Here is the link to Matt Welsh's Ph.D. thesis on SEDA if you have time to read it.
    http://www.eecs.harvard.edu/~mdw/papers/mdw-phdthesis.pdf

    I don't know how that fits into that discussion about performance but this is what I think.

    For me, performance is doing less with less.

    If you use less resources you'll do more.
    If you do less things (don't implement things that are not really used) you'll do more.

    I undestand that doing less that 100% of the specs of an API or a protocol is probably unacceptable to a general purpose project but for confidencial projects there is always something that is superfluous.
  19. We've talked about this at length...[ Go to top ]

    As to Lukasz Kubica's claim that "Tomcat 5 is not well designed inside (in fact it's quite messy ). If i were trying do make nio enabled servlet container, i'd choose jetty." -- please. Have fun ;)

    For interesting reading:

    http://www.mortbay.com/MB/log/gregw/?permalink=servletsMustDieSlowly.html
    http://www.mortbay.com/MB/log/gregw/?permalink=NIOServlet.html

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Shared Memories for J2EE Clusters
  20. 1. Original SEDA [is] crap
    2. Original SEDA async socket [is] crap^2 (that's why i decided to use netty)
    [..]
    6. Tomcat 5 is not well designed inside (in fact it's quite messy ).

    Could you explain a bit why you feel this? Is it the architecture? The APIs? The implementation? Any specifics?

    I looked at some of the SEDA architecture docs and it looked pretty well thought out .. very similar to some things that I happen to know work well ;-) ..

    Peace,

    Cameron Purdy
    Tangosol, Inc.
    Coherence: Shared Memories for J2EE Clusters
  21. Servlets..[ Go to top ]

    In the discussion on servlets here, it was talked about (and in some of the references) how NIO/SEDA is a problem with the current Servlet model. I can see that, and though you can use SEDA or NIO within your servlet for processing, the overall Servlet API isn't designed with that model. (In fact, it was interesting to see the Jetty testing with NIO showing higher latency at low request volumes, given the event processing overhead..)

    Obviously we could get higher performance by revamping the whole model, but that would come with even more complexity.. perhaps making it event driven, queued processing, etc.. does it make sense to build a different API alltogether? It seems like servlets meet a good need.. but at the higher volume end of the table people want more flexability and capabilities.

    I personally like the idea of a different API of some sort.. if we try to jam all of these things into Servlets, we are going to bloat it and add more complexity, and probably still not make it as powerful as it could be.
  22. So the "(like Swing)" part refers to event-based, not easy, right?
  23. You might already know, but I'm writing this message just to make sure people get to the right place. The author (me) is currently working on the Apache MINA project (http://mina.apache.org/), which is a successor of Netty2. I don't support Netty2 anymore, so please upgrade to Apache MINA, the network application framework.