Discussions

EJB programming & troubleshooting: java.lang.OutOfMemoryError on very long time consuming usecase

  1. Hi guys,
    I have a really serious problem so any input from you is greatly appreciated. I am using as an application server JBoss 3.0.0 and Oracle9i as a Database. Changing the app server is not possible.
    I have a very long time consuming process which eventually ends with the exception:

    11:03:51,921 ERROR [LogInterceptor] TransactionRolledbackException, causedBy:
    java.lang.OutOfMemoryError
    11:03:51,921 ERROR [STDERR] java.rmi.RemoteException: null; nested exception is:
    java.lang.OutOfMemoryError
    Embedded Exception
    null; nested exception is:
    java.lang.OutOfMemoryError
    11:03:51,921 ERROR [STDERR] at it.deltadator.common.util.IdGenerator.getId(IdGenerator.java:33)
    11:03:51,921 ERROR [STDERR] at t.deltadator.cdg.be.consuntivo.ejb.ses.ManagerCompetenziazioneBean.saveCompetenziazione(ManagerCompetenziazioneBean.java:146)
    11:03:51,968 ERROR [STDERR] at it.deltadator.cdg.be.consuntivo.ejb.ses.ManagerCompetenziazioneBean.executeCompetenziazioneConsuntivo(ManagerCompetenziazioneBean.java:287)
    .................

    Let me explain you the usecase. I have two tables A and B. In table A there are two columns: DateStart and DateEnd. Let's assume I have a record in table A with DateStart= '1 January 2004' and DateEnd = '1 July 2004'. For each day occuring between these two dates, I have to create a record in table B. So using this example, I have to create around 180 records inside table B (31 for January, 29 for February,..., 30 for June, 1 for July).

    The problem appears when I have hundreds of records inside table A. Imagine that for 1000 records inside table A with DateStart= '1 January 2004' and DateEnd = '1 July 2004' I have to create 180 * 1000 = 180.000 records inside table B!!!
    I can see why JBoss runs out of memory. I have to tell you that for each record inside A is started and ended a transaction. So there is no global transaction.
    I would like to hear from you some ideas about how to implement this usecase. Currently it's using Business Delegate + Session Bean + Entity Beans.
    In my opinion it's better to put everything inside a Stored Procedure and just call that, but the client wants to see the progress (a progress bar with a message like "101st from 200 processed...")and that's why we are using this architecture.
    As a temporary solution we've increased the memory allocated for the JVM but I want to find a better solution.
    Thanks for all your suggestions.

    Sergiu.
  2. Actually the amount of records does not explain the lack of memory, if transactions are kept sort. The problem is the amount of entity beans. Bulk operations like this (as well as report generation) should not rely on entity beans. Entity beans are meant for "small", business logic related operations.

    Try to use the datasource directly on your session bean. I think you should use bean managed transactions in this case (commiting after processing each row in "A").

    If you want to insist on using entity beans, you'll have to take a deeper look into JBoss deployment descriptors, specifying a maximum size to the pool of entity beans. But in the session bean it is still a good idea to manage the transaction yourself (in this particular case).
  3. There is another thing: the best practice for time-consuming operations is the use of MDBs.

    But the progress bar requirement complicates things a bit.
  4. Actually I have proposed a solution like this:
    BusinessDelegate ----> Stored Procedure(pk_record_tableA) so for each record from tableA I call this stored procedure. This way I still can inform the user about the progress of the usecase.
    I know this couples the application with the database but the price is so small if we think to the performance we gain.
    I'm told to stick to CMP for now. Maybe in the future...
    Thanks Andre for your thoughts.
  5. Well, if you use a stored procedure you are giving up CMP anyway (in this particular case), and that's what I meant. Since the business delegate is (probably) a "client-side" class, I believe you are using a session bean to invoke this stored procedure. This session bean should have bean managed transactions.

    It is an acceptable design, but you must document this piece of code/design carefully for it is a break in portability.