I want to retrieve some heavy buffer data from the database and store it using some sort of caching mechanism at the Application server end. So that when web server requests the data, I can check using some field, say it Moedified DAte Time from the database and download if the Date has changed.
Please suggest how to go about it
It will depend solely upon how big your database is.
Also consider the scope of growth of the database in size over a period of time.
I hear people talking too much about caching;)
It would be totaly unwise to cache any data that is frequently modified.
Where in case of lookup data, which is rarely modified , it could be cached for minimising the IO between the DB and the appserver.
If you are planning to query for modification flag when you receive the request, then I guess you care not using caching in optimum way.
Server side cahing should be used mainly to achieve following
1. Avoid expensive database calls when the data is demanded by keeping it ready in memory.
2. To speedup "very frequent" data lookup (typically configuration data) calls when target data remains relatively stale.
The only challenge while designing such cahe is
1. To identify such data set which is stale but very frequently required.
2. To synch up this data
Very common solution to keep data in-synch is polling. You need to have Data Cache, Poller and Data Fetcher. Ofcourse now you know the rest.
I have suggested you very basic cahcing mechanism that you can develop and support very easily.
Hope this meets your requirement.