Discussions

News: J2EE Article: Load Control Filter

  1. J2EE Article: Load Control Filter (14 messages)

    The following article is based on real world experience. The text discusses the problem of improving web application scalability without refactoring existing business logic.

    We present the design and implementation of a servlet filter that restricts the load each user can put on a web application. It prevent a downward performance spiral where well-intentioned users drag an already overloaded server to its knees.

    The full text of the article is available here:
    http://cocoonhive.org/articles/LoadControlFilter/load_control_filter.htm

    What other techniques would you use to improve scalability without refactoring?

    Cheers,

    Ivelin

    Threaded Messages (14)

  2. You are assuming that the user only cares about the last rquest they have made. Users also open the links on a web page in new browser windows instead of the same window and they do care about all the requests they have made. In this case, their requests won't get served, except for the most recent one.
  3. The most recent one....[ Go to top ]

    Users also open the links on a web page in new browser windows instead of

    > the same window and they do care about all the requests they have made.

    Very true. Especially, if large parts of the application are slow you will
    use other parts of the site again.

    > In this case, their requests won't get served, except for the most recent
    > one.

    And of course there are a lot of situations, actually almost ALL situations that deal with transactions, where you would want not the most recent request to be properly executed but the FIRST request.
  4. J2EE Article: Load Control Filter[ Go to top ]

    See for example Traffic filter in JSOS:
    http://www.servletsuite.com/servlets.htm
    or
    http://www.servletsuite.com/servlets/trafficflt.htm

    TrafiicFilter lets you prohibit access to your site for clients with excessive hits


    Dmitry Namiot
    Coldbeans
  5. "In the example, you changed your mind twice and sent three requests to the server." I disagree with this. I do this all the time, but it doesn't mean I've changed my mind. I use tabbed browsing in Mozilla, and it is my standard practice to open multiple links in the background from all of the links on the page.
  6. Intentional multi-click[ Go to top ]

    Notice that the filter is flexible enough to where it can be configured with a wildcard expression, which links should be filtered.
    When there are multiple links in a page that the user is expected to open at once, the filter can be configured to avoid blocking.

    Keep in mind that a news portal and a blog site are very different web applications from online banking or bill payment. Users of business applications rarely follow multiple workflows at once.
    While opening multiple pages at once might be a "nice to have" feature, responsiveness and reliability are "must have" features.
    A matter of balancing.

    Ivelin
  7. Even if one accepts the premise that, in the "three clicks close together" scenario that you really are only interested in the last message, the proposed filter design has some problematic impacts on scalability of your server:

    * Even if the user is not interested in
      the first two responses, the server is
      going to complete the execution of them
      anyway, so the CPU time is still going to
      get "wasted". The nature of HTTP makes it tough
      to deal with this, but it would be feasible
      to have long-running transactions periodically
      poll for a flag variable in the session that
      says the user has submitted another request
      and no longer needs the first one to be
      completed.

    * While a request is in the queue, it is tying
      up a request processing thread in your servlet
      container. Not only will it not be serving the
      user in question, it will not serve any other
      user either. This has the effect of limiting
      the number of simulataneous users your app can
      support. To say nothing of what malicious users
      can do if they know your app is using this technique.

    Craig McClanahan
  8. Negative Impact On Server Scalability[ Go to top ]

    Craig, on your first point of the server processing the requests anyway. The server will process the requests only to the point of the filter. The long running service will not be run except for the currently running thread. I don't see where CPU time is being wasted other than the current request. Having requests poll the session for imcomming reqests still wastes CPU time. Say I make 10 requests to a servlet, one following the other, just before the previous request completes. I end up wasting CPU time for all of the requests except for the last. The filter servlet would process 2 of the 10 requests.

    On your second point, your right, a connection is being used for each request, specifically the one in the queue and the one currently being processed. But I don't see how this is worse than many requests coming into the server and all of them being processed simultaneously? Each request beyond the one in the queue will be 'return'ed immediately freeing up a socket connection.
  9. J2EE Article: Load Control Filter[ Go to top ]

    Hi,

    For me it is a nice filter. Only a small remark: why the 'getSynchronizationObject' should be synchronized for every request of every session? I think synchronization on a session within this method will be enough.

    Regards

    Alex
  10. J2EE Article: Load Control Filter[ Go to top ]

    Hi,

    >
    > For me it is a nice filter. Only a small remark: why the 'getSynchronizationObject' should be synchronized for every request of every session? I think synchronization on a session within this method will be enough.

    I dont know if it is the reason in this case since I didnt write the code, but in some servlet containers (tomcat 5 (still?)) it is not possible to synchronize on the actual session object, since there are no guarantees that the actual session implementation instance is the same for repeated calls to getSession().
  11. J2EE Article: Load Control Filter[ Go to top ]

    Is it true ? For me it sounds unbelievable besides the case when one tells about a distributed container. How on the earth would it be possible to synchronize session attribute access/manipulation dependent operations ? Only by using ServletContext or a stuff alike ? Sorry if I am asking a silly thing )
  12. J2EE Article: Load Control Filter[ Go to top ]

    Is it true ? For me it sounds unbelievable besides the case when one tells about a distributed container. How on the earth would it be possible to synchronize session attribute access/manipulation dependent operations ? Only by using ServletContext or a stuff alike ? Sorry if I am asking a silly thing )


    No, it sounds silly, and it seems like this was a bug in tomcat and nothing else: http://nagoya.apache.org/bugzilla/show_bug.cgi?id=17615. The issue is real though, since AFAIK the servlet specification does not mandate that the same instance is returned. If I am wrong, please correct me.

    Br - J
  13. J2EE Article: Load Control Filter[ Go to top ]

    Your are right. I could not find any remark about 'the same session instance' as well. But it would be logical to have such thing. Is it the only workaround to manage trouble-free session attribute access (when appropriate) to synchronize on some global container instance (ctx, filter, own singleton etc.) ? Probably in the filter case one could define HttpSessionListener and put the synchronizationObject in a session on sessionInit(). But unfortunatelly there is another question: when a session listener will be invoked before or after a filter...
    I appreciate a lot if somebody could throw a light on the subject.

    Thanks
  14. J2EE Article: Load Control Filter[ Go to top ]

    Ok, it is a good practice to limit the number of requests that users can make but, to my mind it would be better to inform users about the load the server has in case of overloading instead of having them going crazy waiting for their requests :) . Is there any filter that can detect server load? I used to do this by checking the number of threads on JServ but that does not work on OC4J. Any standard way of doing this?
  15. Very good Article[ Go to top ]

    Very good Article. Based on the other users response it can
    be tweaked/configured to accomodate different environments

    For example the following can be designed to be configured
    a) the max number of active requests per user
    b) Based on the total load on the server the number in a) can be varied by
         some formula
    c) Maybe provide a weight to requests and requests with more processing
         times can have a bearing on the number in a)