> general lack of understanding of this in the Java development community
I concur with James. You can look to use Hibernate StatelessSession, Ibatis RowHandler and Ebean ORM has QueryListener that provides Persistence context per Object graph. These can be much more efficient memory wise for processing large queries on a per object graph basis.
Your right there with me but this is just the tip of the iceberg. Consider a simple read-only web service that pulls from a database. The following occur in creating returning a response:
1. Query the database
2. Build an in memory representation of the response
3. Transform that data into XML
4. Send the bytes across the socket
Assume that these steps take 2 seconds, 1 second, 1 second, and 1 second respectively.
In most approaches that I see, thes steps all take place sequentially so it takes 5 seconds total.
If, however, you start step two as soon as you have enough detail to build the first object and start tranforming and transmitting as soon as possible, you can reduce this time even on a single core. The reason is that the db and the client are generally on a different machine so instead of your service sitting there like a bump on a log while in IO wait, you are building your objects. If your client is built in a similar way, the user (assuming there is one) might even see some part of the response before the query finishes.
This is often easier said than done because the libraries we use often are not designed in this manner and there are some downsides to the approach but I think people need to be more aware that the way things are usually done is not the only way to do it. At the very least we should all understand how the common approach works and why it doesn't support large messages very well.
