i am facing a typical problem in the resultset i get around 40,000 records which i need to display in batches of 10,000 recs each. but while displaying such a huge data on browser ,we r also processing the data i.e. calculating the totals and subtotals which is making it slower and slower
can anyone suggest a better way or design for this?

do you just get accessed to the data by the jbdc?or something else?

You could implement a custom Value List Handler, see the J2EE pattern at: http://java.sun.com/blueprints/patterns/ValueListHandler.html

The retrival process can be made faster if u could create an index on the columns where uve got to perform computations.

I think that, if you are trying to display 10000 records in one page, that will be a big problem right there, regardless of what else you are doing. The amount of time it takes to generate the HTML and transmit it to the browser will be huge. If you need to do this, I would make sure to not buffer the output until it's all prepared, but instead to send back output as the page loads on the browser. That way the user at least sees the information coming back as it is transmitted and can tell it is working.

Also, a couple of other things:
1) As someone else said, make sure the indexes are properly created on the table. You should be able to do something equivalent to an "explain plan" to see how the DB is handling the query and make sure that it is being handled optimally. With that many records, you're probably going to get a table scan, but hopefully the sorting will at least be helped.

2) Don't build the HTML results in a string and then display them. If you do a lot of string building, make sure to use a StringBuffer class instead. If you are doing lots of string building with String, you will probably notice the app get slower and slower as you go, since it has to do lots of memory reallocations for the string.

Hope this helps.

Mark

Displaying 10,000 records in one page is just plain wrong for several reasons:
1 - As you noted, its darn slow.
2 - Browsers aren't that good with huge tables, and they become very unstable with that sort of data (my experiance, correct me if I'm wrong)
3 - As you introduce more functionality it can only get slower.

My advice, try to chunck the data. Provide some sort of next, previous, first, and last functionality. All web interfaces for OLAP engines for example has that becuase the amount of data they deal with is huge. plus, who on earth is capable of browsing 10,000 records at a time let alone make any sense of them ???

Hello there

Add my point also to this

have Wizard for this data presentation (Some one said about having the navigation link Next > Previous some thing like this)

Will caching of data will reduce your response time ??

Any comment from Gurus ?+

Cheers
M