Hello everyone I have a number of question regarding J2EE development.
In particular these relate to the use of CMP Entity beans.

I have been working on an application used as a service
provider to mobile phones. This application has a number of
requirements:
  - It has to be highly scalable and reliable. In
    theory it could be processing requests from millions of
    users (mobile phone devices).
  - Highly modular, it has to be easily adaptable to the
    environments of our customers (these are usually mobile
    phone operators but can also be third-party content
    providers). So for example we should be able to support
    multiple databases, and application servers.

My company has decided to use J2EE because :
  - It is a standard
  - It has great support for clusterring (which we need for
    performance and robustness)

We have developed an initial prototype and are now running
some tests to see how to improve it and also discover
problems we may have. The problem I am concerned with in
this post relates to data persistence;

We use CMP for object persistence, this is hidden behind
stateless session beans, which are accessed by our web-based
administration interface and a number of servlets which
are invoked by the phones.

The problems we are running into seem to stem from how we
have been using our CMP Entity Beans. Some of these can have
many intances, for example we have a SIM Entity bean used
to represent a mobile phones sim. Retrieving just the ones
we need (and therefore avoiding running out of memory is
becoming a nightmare). For example when I want to list the
sims on my admin webinterface, I would like to retrieve only
a 20 or so (the ones actually visible on the webpage) the
list of sims will also most probably have been filtered so
that only a subset of these are applicable.

We have struggled to find a suitable pattern for this and
have rolled out our own, based on providing methods in the
stateless session beans such as:

  List SimManagerSessionBean.getBeans(List filters,
                                      int offset,
                                      int limit);

Where filters is a list of filters used to find out which
SIMs are applicable, offset and limit are used to retrieve
only a subset of the applicable SIMS. This method returns
Data objects representing SIMs.

The problem I am having with CMP is that there does not seem
to be a generic way of specifying limit and offset.
Therefore we use direct JDBC statements to retrieve the
data. We wrap the creation of the JDBC statements with a
helper class to help isolate JDBC but it is still not very
elegant.

Anyway I decided to take a step back and revaluate our
approach and have been looking at Hibernate as an
alternative to CMP and Spring as tool to help structure our
code. My current plan is to:
  1- Build a simple stand alone Hibernate application and
     see if it can do what I need.
  2- Replace my CMP with Hibernate
  3- Read more about Spring and see if I can use it to any
     benefit in my application.

The reason I am looking into hibernate is that it seems to
provide finer control and seems to allow me to do the stuff
I mention above. I do not want to have 2 ways of accessing
my persistent data (CMP and raw JDBC).

Here is some of the question I have running in my head now:
  - Can Hibernate replace CMP especially given my clustering
    requirements?
  - Can Hibernate's use of transactions be integrated into
    my application's transaction that come from my J2EE
    application server environment?
  - Could it be that I am missing the point completely and
    CMP will do the job just fine?
  - How do people deal with these problems? (I can't seem to
    find any patterns for this sort of problem and no one
    seems to mention them).
  - What benefits would using Spring give me?

I would appreciate any feed back and comments from this
community.

Regards,
Hani