Using in-memory database for data processing

Discussions

Performance and scalability: Using in-memory database for data processing

  1. Using in-memory database for data processing (4 messages)

    Dear Experts, I've searched for information on what seems to be a simple question but cannot find an answer. My question is, would it be a good idea to use an in-memory database, such as HSQLDB or H2, for data processing, instead of creating complicated structures of Maps and Lists? In other words, what do you think of the idea of using an in-memory db, just for the duration of your data processing, to store your calculations and provide an easy way to query the results? Just throwing the idea out there, I'd like to know what you think, if you've ever tried something like this.. let me know if you need more details.. Thanks in advance Frederic
  2. Hi, Frederic. It's good idea, we use such approach in some apps. But u need an architecture analisys and stress testing in every case to understand what approach to use.
  3. My question is, would it be a good idea to use an in-memory database, such as HSQLDB or H2, for data processing, instead of creating complicated structures of Maps and Lists?
    Depends on if you are wanting to use SQL or Java to work with the data. If SQL, then check out TimesTen. If Java, and you are clustering, then check out Coherence. Peace, Cameron Purdy Tangosol Coherence: The Java Data Grid
  4. Thanks Cameron and everyone for your responses. I ended up trying with H2, which worked out nicely, and JoSQL, which worked out even nicer and is my current solution. Cheers, Frederic
  5. Hi Frederic, Although I'm new to this domain and can't say am expert..... However feel this is good idea..we have used this for certain complex logics in project... I think , it makes amendments/changes and bug rectification easy specially after deployment.. But not sure how it will affect performance if used for all kinds of data processing...probably a database experts can have their say on it..