Hi, we've an application that fetches lakhs of records from one database and stores/inserts into another database by some intermediate logic after data fetch. The database is available in SQL Server. The application is using Java and iBatis to connect to the database. During fetch, it is giving the below exception:
com.microsoft.sqlserver.jdbc.SQLServerException: The system is out of memory. Use server side cursors for large result sets:Java heap space. Result set size:115,179,813. JVM total memory size:66,650,112.
I understood that we're supposed to use Java and simple queries only to use small data sets. For larger data sets, it is better to use ETL tool or PL/SQL procedures and cursors (external).
Is there any way to achieve this in Java using multi threading? I'm suspecting that duplicate records may be inserted if multithreading is used in this case which is not acceptable. Please advice. Thanks.