Discussions

EJB design: Bulk data handling!!!

  1. Bulk data handling!!! (6 messages)

    Hello,

    We are in the process of developing a web based J2EE application. In this application we have large reports with a very huge data (possibly a million records!!!). And there are 5-6 different types of reports which will show different data for different user. And also, these reports' contents changes frequently.
    Can anybody guide me how to design that part?? Can I use EJBs to fetch that much of data and do calculations on that and than send it to presentation layer?? And if EJB is appropriate, what kind of EJB will be suited for this work and why??

    Thanks in advance,
    Pankaj.

    Threaded Messages (6)

  2. Bulk data handling!!![ Go to top ]

    Hi,

    I would use a stateless session bean that uses Data Objects to fetch the data from the db with straight JDBC calls. If the contents of your reports don't change that frequently, but you want to be able to fetch them often, you can also cache the (processed) data in the bean, this will increase performance but is not very applicable if the data changes rapidly or if there are not many requestors of the data. A complicated matter is that the data id different based on the user, I would still opt for a stateless beans though.

    Maybe another option is to use some kind of scheduler to generate the reports for all users in the background, so they are pre-compiled and ready when the user requests them. This heavily depends on the amount of users though.

    Hope this helps,

    JOost.

  3. Bulk data handling!!![ Go to top ]

    Pankaj,

    I would seriously consider using a Reporting engine to design and execute the reports (Actuate, Crystal Reports, etc.) You then could design your own front-end to access and run these reports, using a stateless session bean to connect to the report engine.

    Tim
  4. Bulk data handling!!![ Go to top ]

    How about a stateful session bean that will manage a JDBC scrollable result set. This will allow each user to navigate through a large result set.

    Pranab
  5. Bulk data handling!!![ Go to top ]

    Hi,

    I've tried this, but the problem is, you have to define the ResultSet as member of your bean. This, infact is a problem, because the ResultSet is not Serializable....other problem is, that you need to have one Transaction, that is open while getting the pieces of data.

    Any suggestions?

    Lutz.
  6. Bulk data handling!!![ Go to top ]

    Hi,

    Maybe you can keep track of your cursor position in the ResultSet and open it on the correct position on the subsequent calls of the client, this way you circumvent sertialization and transactional issues..

    Joost.
  7. Bulk data handling!!![ Go to top ]

    The SFSB does not need to be transactional if you are only doing the SELECT query. To alleviate the serialization problem, you could store the result set in a Hashtable managed by a singleton. The string name of the result set will be stored inside the SFSB.

    Pranab