My project services different group of users. Due to the complex processing, the response time is not good especially when one user are keep firing transactions. This will drain out the resource and other users may experience long waiting-time.
So we are thinking about implementing some kind of Service Level Agreement to regulate the traffic. Each user will be given a TPS(transaction per second) value. If the value is exceeded, the successive method call may be hung.
The problem is that, the TPS will reduce the system throughput. For example, even only one user is using the system, he still can not exceed the TPS. This is wasting the resource. How to get a balance between? That is , to prevent some user from starving and to maximize the system through as the same time?
What experiences & technologies should be followed to address the problem?
Or any other good points?
If the transaction's time excedes SLA, it's OK to stop it, rigth? No matter how many users are using the system...
Unless you have complicated SLA schemes, depending on the kind of operation, user, time of the day, blah blah. Usually, SLA are hard to represent, I think, just because there are SO many, and they all affect the whole system.
Once a transaction begins we can not stop it. What we will control is the Transaction Per Second (TPS) for each user.
As you said, the SLA is too complext to define one. So we just define the simplest one, which is TPS as mentioned above.
TPS alone is not hard to implement. The problem is that, if we strictly follow the TPS, system capacity may not be fully utilized. How to find a balance point between TPS and system throughput.
For example, if there are ten users all together, each user is allowed 1TPS since the system capacity is 10 TPS.
What if only one user is active ? Due to the TPS, system utiliztion is only 10%. That's not good.
Sorry, man. Cannot help you there. Zero experience with this kind of stuff...
Load test to determine the systems ability to handle heavy load. Determine how many TPS you want to limit the system to, say this # is x (this number may be a dynamic and may be updated using heuristic) during the time when system resources are available due to non-peak utilization, although the single user TPS has exceeded determine how much of the available TPS you want to extend to. And use an algorithm to control access based on these three factors. In any case the limits cannot to rigidly enforced because as you said you cannot interrupt a transaction once it has started. If your business model allows you to put a hard limit on per user TPS, go for it irrespective of whether the current load conditions permit extensions or not. If not then consider the current load into the mix for determining whether to suspend the service or not. Hope this helps.
thanks men !