Using a BLOB data type in a CMP Entity Bean

Discussions

News: Using a BLOB data type in a CMP Entity Bean

  1. Using a BLOB data type in a CMP Entity Bean (12 messages)

    You cannot directly map a java.sql.Blob field type in a CMP entity bean to a BLOB data type in the database and use byte to manipulate the BLOB data type. Debu Panda paints the picture for a work-around.

    Read Using a BLOB data type in a CMP Entity Bean

    Threaded Messages (12)

  2. Performance[ Go to top ]

    I tried this solution before read your blog and I think that don't scale very well. To picture files, with about 1 megabyte it can seems work, but if you want use BLOB seriously (more than 20 megabytes) it will consume too many memory and will be slow.

    First of all, whole file will be put in byte array memory. If you use Remote Interfaces, or even local (in some appserver configurations) , can occur copy of the byte array, allocating twice of the space of the blob. All this proccess will take some time.

    I preffer make single JDBC calls for that, and stream the file with a InputStream's, or make it by chunks of data, without entity beans. Other option is save the image in the filesystem temporary space, and use some extension of your database to load the file directly from the filesystem, if both runs in the same machine (or share a common drive).

    An the best solution, in my opinion, don't save the file in database. Keep them in filesystem. Images, like the example, will be faster if you point them directly from the browser, avoid passing by a servlet that retrieve them from the database.
  3. BLOB is good for us[ Go to top ]

    On my current project we have to store a whole wack (TM) of images for a few thousand models in our system. Rather than come up with a way of synchronizing the images across the file systems of 5 different tomcats and 3 application servers, we opted to simply store the images as BLOBs using CMP (CMP could easily be refactored to JDBC for streaming images back to browsers in our app). This makes managing the images a none-issue as there is only a single source for them (the database).

    This used to be a bit of a pain because we initially relied on OCI drivers to get around the 4K problem. I just switched to the 10g THIN drivers after reading this thread and everything is working fine; hurray!

    Your mileage with this kind of solution may vary, but it's way easier for us to go this route than to solve the problem of distributing freshly uploaded content (which comes fast and furious in our app) across a host of file systems.
  4. I agree with George to the extent of using JDBC to write to DB.

    When it comes to scaling and performance in general, using CMP beans to write directly to the DB may not be the most correct design. Granted, scaling/performance depend on the no. of concurrent requests.

    I would still prefer to write to the DB using JDBC and arrive at the optimal chunk size to write that many no. of bytes to the DB in a loop.
  5. Liferay JDBC Driver[ Go to top ]

    You can also use the Liferay JDBC driver, and use CMP without any changes to talk to long varchars. Columns over 4k break for DB2 and Oracle when you use CMP.

    See http://www.liferay.com/documentation/development_databases.jsp#oracle

    We had to write this a few years ago when we used Entity beans, then we switched over to Hibernate, but still had to use CMP.

    It's not an exact fix for blobs, because we use oracle's long varchar, and serialize the object into a string and back.

    It tests the lenght of the setString, and if it's longer than 4 k, uses setAsciiStream (same solution for a CLOB posted by Oracle years ago).
  6. Liferay JDBC Driver[ Go to top ]

    We have (fixed) worked around the 4K limit in the JDBC driver and OC4J 9.0.4 works great with size higher than 4K for CMP entity beans with BLOB/CLOB in Oracle database. Also there is a patch available for OC4J 9.0.3.

    regards
    Debu Panda
    Oracle
  7. Liferay JDBC Driver[ Go to top ]

    What if you use Hibernate and not CMP? Does the new Oracle JDBC driver fix it automatically? Or is it tied to OC4J's CMP impl?
  8. Liferay JDBC Driver[ Go to top ]

    "What if you use Hibernate and not CMP? Does the new Oracle JDBC driver fix it automatically? Or is it tied to OC4J's CMP impl? "

    This was an issue with older versions of THIN drivers and has been fixed in 10g THIN driver. If you are using an older version of JDBC driver you can use OCI driver that do not have this limitation.

    -Debu Panda
    Oracle
  9. Two bad Ideas in one...
    Using BLOBS, and using Entity Beans. :)
    are we sure this isn't another anti-pattern?
    its hard to test & doesn't scale well. (for any blob of large size or for performance)

    anyone have good (real world) experience with using BLOBS with Entity Bean?

    who would actually do this?
  10. This works fine[ Go to top ]

    Two bad Ideas in one... Using BLOBS, and using Entity Beans. :)are we sure this isn't another anti-pattern?its hard to test & doesn't scale well. (for any blob of large size or for performance)anyone have good (real world) experience with using BLOBS with Entity Bean?who would actually do this?
    I use exactly the same schema for yet already the 3-rd project and I can assure you - it works fine. Two of that projects are in production for about a year. No problem.

    Although I've stored not a very big data such a way - less than 1 mb each, I was just considered this way as *very convenient* approach persisting those java object instances, which actual classes are known only at runtime. Or objects with extremely large amount of fields (User Pallette). I've just serialized their VOs into the BLOB field.

    Of course, this technique is not the rerfect choice if you're doing video streaming site, even Java rather not a right choice here ... but for office applications this more than enough.

    Sincerelly,
    --
    Mike Skorik
    www.100kSolutions.com
  11. Jason,

    Think about a company which has more than 10,000 employees and they want to store pictures of the images, how difficult to manage these pictures when you do not save this in database. So definitely BLOB is not a bad idea. We have thousands of database customers who use BLOB

    These pictures do not have to a very high resolution and always less than 1MB. Presonally I know for sure 3-4 of our application server customers who have been using CMP with BLOB/CLOB successfully in production environment. However I'm not privileged to reveal their names.

    I just did a google on +CMP +BLOB and this yielded me 3630 results and these were mostly questions on mailing lists so it appears that there have been a lot of interest with CMP and BLOB.

    -Debu Panda
    Oracle
  12. I dont think its a good design, for a similar kind of requirement, We have used the file upload utility to upload the large image files ,directly to the server file system with adding the time stamp to the name of the file. the URL of this uploaded files is then stored in the database table . At the time when u want to fetch the image file and display it on webpage ,just fetch the URL from the database and use it.
    Thanks
  13. I dont think its a good design
    I used byte Arrays 18 months ago for storing Excel Spreadsheets in an Oracle table (8.1.7). As the standard was to use CMP this is the only way I could get it working (under bea Weblogic 6.1 sp3). After much searching on newgroups etc to no avail I managed to get it working after some suggestions from bea Support.

    Whilst the Excel files weren't huge, it worked and scaled really well.

    In the real world working comes first and speed of delivery (hence CMP). Performance can be addressed in system test.

    My two Euros worth.