The following article is based on real world experience. The text discusses the problem of improving web application scalability without refactoring existing business logic.

We present the design and implementation of a servlet filter that restricts the load each user can put on a web application. It prevent a downward performance spiral where well-intentioned users drag an already overloaded server to its knees.

The full text of the article is available here:
http://cocoonhive.org/articles/LoadControlFilter/load_control_filter.htm

What other techniques would you use to improve scalability without refactoring?

Cheers,

Ivelin

You are assuming that the user only cares about the last rquest they have made. Users also open the links on a web page in new browser windows instead of the same window and they do care about all the requests they have made. In this case, their requests won't get served, except for the most recent one.

> the same window and they do care about all the requests they have made.

Very true. Especially, if large parts of the application are slow you will
use other parts of the site again.

> In this case, their requests won't get served, except for the most recent
> one.

And of course there are a lot of situations, actually almost ALL situations that deal with transactions, where you would want not the most recent request to be properly executed but the FIRST request.

See for example Traffic filter in JSOS:
http://www.servletsuite.com/servlets.htm
or
http://www.servletsuite.com/servlets/trafficflt.htm

TrafiicFilter lets you prohibit access to your site for clients with excessive hits


Dmitry Namiot
Coldbeans

"In the example, you changed your mind twice and sent three requests to the server." I disagree with this. I do this all the time, but it doesn't mean I've changed my mind. I use tabbed browsing in Mozilla, and it is my standard practice to open multiple links in the background from all of the links on the page.

Notice that the filter is flexible enough to where it can be configured with a wildcard expression, which links should be filtered.
When there are multiple links in a page that the user is expected to open at once, the filter can be configured to avoid blocking.

Keep in mind that a news portal and a blog site are very different web applications from online banking or bill payment. Users of business applications rarely follow multiple workflows at once.
While opening multiple pages at once might be a "nice to have" feature, responsiveness and reliability are "must have" features.
A matter of balancing.

Ivelin

Even if one accepts the premise that, in the "three clicks close together" scenario that you really are only interested in the last message, the proposed filter design has some problematic impacts on scalability of your server:

* Even if the user is not interested in
  the first two responses, the server is
  going to complete the execution of them
  anyway, so the CPU time is still going to
  get "wasted". The nature of HTTP makes it tough
  to deal with this, but it would be feasible
  to have long-running transactions periodically
  poll for a flag variable in the session that
  says the user has submitted another request
  and no longer needs the first one to be
  completed.

* While a request is in the queue, it is tying
  up a request processing thread in your servlet
  container. Not only will it not be serving the
  user in question, it will not serve any other
  user either. This has the effect of limiting
  the number of simulataneous users your app can
  support. To say nothing of what malicious users
  can do if they know your app is using this technique.

Craig McClanahan

Craig, on your first point of the server processing the requests anyway. The server will process the requests only to the point of the filter. The long running service will not be run except for the currently running thread. I don't see where CPU time is being wasted other than the current request. Having requests poll the session for imcomming reqests still wastes CPU time. Say I make 10 requests to a servlet, one following the other, just before the previous request completes. I end up wasting CPU time for all of the requests except for the last. The filter servlet would process 2 of the 10 requests.

On your second point, your right, a connection is being used for each request, specifically the one in the queue and the one currently being processed. But I don't see how this is worse than many requests coming into the server and all of them being processed simultaneously? Each request beyond the one in the queue will be 'return'ed immediately freeing up a socket connection.

Hi,

For me it is a nice filter. Only a small remark: why the 'getSynchronizationObject' should be synchronized for every request of every session? I think synchronization on a session within this method will be enough.

Regards

Alex

>
> For me it is a nice filter. Only a small remark: why the 'getSynchronizationObject' should be synchronized for every request of every session? I think synchronization on a session within this method will be enough.

I dont know if it is the reason in this case since I didnt write the code, but in some servlet containers (tomcat 5 (still?)) it is not possible to synchronize on the actual session object, since there are no guarantees that the actual session implementation instance is the same for repeated calls to getSession().

Is it true ? For me it sounds unbelievable besides the case when one tells about a distributed container. How on the earth would it be possible to synchronize session attribute access/manipulation dependent operations ? Only by using ServletContext or a stuff alike ? Sorry if I am asking a silly thing )

No, it sounds silly, and it seems like this was a bug in tomcat and nothing else: http://nagoya.apache.org/bugzilla/show_bug.cgi?id=17615. The issue is real though, since AFAIK the servlet specification does not mandate that the same instance is returned. If I am wrong, please correct me.

Br - J

Your are right. I could not find any remark about 'the same session instance' as well. But it would be logical to have such thing. Is it the only workaround to manage trouble-free session attribute access (when appropriate) to synchronize on some global container instance (ctx, filter, own singleton etc.) ? Probably in the filter case one could define HttpSessionListener and put the synchronizationObject in a session on sessionInit(). But unfortunatelly there is another question: when a session listener will be invoked before or after a filter...
I appreciate a lot if somebody could throw a light on the subject.

Thanks

Ok, it is a good practice to limit the number of requests that users can make but, to my mind it would be better to inform users about the load the server has in case of overloading instead of having them going crazy waiting for their requests :) . Is there any filter that can detect server load? I used to do this by checking the number of threads on JServ but that does not work on OC4J. Any standard way of doing this?

Very good Article. Based on the other users response it can
be tweaked/configured to accomodate different environments

For example the following can be designed to be configured
a) the max number of active requests per user
b) Based on the total load on the server the number in a) can be varied by
     some formula
c) Maybe provide a weight to requests and requests with more processing
     times can have a bearing on the number in a)