Create DOM - Performance issue

Discussions

XML & Web services: Create DOM - Performance issue

  1. Create DOM - Performance issue (3 messages)



    My requirement is - Pull a set of records from the database. Score it based on certain criteria and sort it and just display first 20.

    How I am doing is following

    1. Fetch the records from the database.
    2. Create a Document(DOM) from result set
    3. While creating do the scoring also and add score as tag element.
    4. Apply XSL on the DOM to sort it based on the score and "other tag element" which is text based
    5. Since we want to display 20, remove the rest of it


    Now if there are 800 rows in the result set, it is really taking time like 6 secs. I do notice that 3.5 secs are being taken in executing the query on which my database team is working but really it takes rest 2.5 seconds from step 2 to 5. I notice that step 2 is second one which takes about 1.5 sec.

    Is there a way to improve the performance? Is there a better difference approach?

    I am using resin 2.14 with db2. XML and XSL parsers are of resin only.

    Please share your thoughts!

    Thanks
    Jas

    Threaded Messages (3)

  2. Create DOM - Performance issue[ Go to top ]

    I would do the sorting before creating the DOM tree:
    1. Fetch from database into a Collection
    2. Sort using Collections.sort(List, Comparator)
    3. Create DOM tree
  3. Create DOM - Performance issue[ Go to top ]

    You may want to try this to improve your performance. We did a similar thing but we were workiong with tables that had tens of thousands of records. We were able to increase ou performance fom minutes to sub second performance. Since you are displaying only 20 rows at a time to the user. Dont go to to the database and get all 800 rows from your table and keep the user waiting to retrieve your data and create you objects. First do initial read to get the total count of records in the table we were querying.
    Long count = "select count(*) as count from <TABLE NAME> Where ?
    Then use this number as an begining and ending index to perform smaller incremental Selects when user wants the next set of 20.

    You can tweak the ammount of records to retrieve to improve your perfomance.
  4. Create DOM - Performance issue[ Go to top ]

    You may want to try this to improve your performance. We did a similar thing but we were workiong with tables that had tens of thousands of records. We were able to increase ou performance fom minutes to sub second performance. Since you are displaying only 20 rows at a time to the user. Dont go to to the database and get all 800 rows from your table and keep the user waiting to retrieve your data and create you objects. First do initial read to get the total count of records in the table we were querying.
    Long count = "select count(*) as count from <TABLE NAME> Where ?
    Then use this number as an begining and ending index to perform smaller incremental Selects when user wants the next set of 20.

    You can tweak the ammount of records to retrieve to improve your perfomance.