PERFORMANCE DEGRADATION OF 10G ON WINDOWS & LINUX WITH DUAL CPU

Discussions

Performance and scalability: PERFORMANCE DEGRADATION OF 10G ON WINDOWS & LINUX WITH DUAL CPU

  1. PERFORMANCE DEGRADATION OF 10G Application Server on WINDOWS and LINUX with dual CPU machines observed when load tesing a J2ee application with webload.

    1.0 Test environment and settings

    Test environment constitutes of load generating tool, application server and Database server isolated using hub and connected by 100Mbps LAN network.

    1.1. Load generating tool
    Web load 6.0 is chosen to simulate the required load.

    1.1.1 Software:
    Web load 6.0 is used for generating load it’s a RadView product. It can simulate load up to 300 concurrent users. It has unlimited load generators. It can generate reports detailed and customized.

    1.1.2 Hardware

    Web load 6.0 is installed on Hp-Compaq system with Windows 2000 OS, 2.6 GHz intel P4 processor and 2GB RAM.

    1.1.3 Settings:
    1. Spawning option enabled.
    2. Perfmon enabled to capture load tool performance

    1.2. Application Server
    Latest Oracle 10g application server is used as middle ware.
    1.2.1 Software:
    Oracle 10g (9.0.4.1) is used as Application Server to run the etrans application.

    1.2.2 Hardware:
    10g Application server is installed on Windows2003EE OS, 3.2 GHz Xenon Processor and 8GB RAM.

    1.2.3 Settings:
    1. JVM settings:
    i)–Xms1024m –Xmx1536m
    2. Connection leakages -Djdbc.connection.debug=true
    3. Garbage collection behavior of VM -verbose: gc -XX:+PrintGCTimeStamps

    Http Server settings:
    1. Maximum Requests Processed Simultaneously (maxclients)– 1000
    2. Request Timeout (seconds) – 600
    3. Connection Timeout (seconds) - 300

    1.3. Database Server
    Oracle 9.2 Release is used as database.

    1.3.1 Software
    Oracle 9.2.0 Release server is installed on Windows2003EE OS, 3.2 GHz Xenon Processor and 8GB RAM.

    1.3.2 Hardware
    AS-PE2850 – DELL™ PowerEdge™2850
    Rack Mount Server
    with Intel® Xeon™ Processor 3.2GHz/1M
    1.3.3 Settings
    SGA - 4GB
    RAM – 8GB

    Total System Global Area 4153484968 bytes
    Fixed Size 456360 bytes
    Variable Size 1468006400 bytes
    Database Buffers 2684354560 bytes
    Redo Buffers 667648 bytes


    -------------------------------------------------------

    Observertions

    1. During single CPU studies at various userloads

    We observed a steady CPU usage @ around 50 %.

    2. During Dual CPU studies at various loads

    We observed after 10 to 15 minutes CPU touch 100 % and remain there till the end of the test.

    Could any body tell us why this behaviour?

    Ours is a typical j2ee application.
  2. well my friend , you are using all the right combinations of parameters here except for one thing...your're missing a a key diagnositic tool which can help you disgonse your j2ee environment, i recommend to use *PerformaSure* a tool from Quest S/w ...we are using it as part of QA and load testing ....it does help in situation like these..
  3. PERFORMANCE DEGRADATION OF 10G Application Server on WINDOWS and LINUX with dual CPU machines observed when load tesing a J2ee application

    100% CPU utilization doesn't present a problem if the application delivers required throughput. If it is not, you have identified that you have a problem.

    Such diagnostics is one of the natural outcomes of load testing. You need to set up a profling environment using one of the profilers (JBrobe, OptimizeIt, JProfiler etc) and re-run load tests under the profiler to identify bottlenecks. After the issue is fixed you have to re-run load tests to ensure that the problem was resolved. You may need to repeat the iteration above till required performance is reached.

    Regards,

    Slava Imeshev

    P.S. Shouting makes performance degradation worse by design.
  4. Maximum Requests Processed Simultaneously (maxclients)– 1000
    this is too high. in this case you can have 1000 + 200 threads in your JVM; you should not have more than 300-500 threads in JVM under normal conditions

    default value for "concurrent HTTP requests" is 64-128 requests / threads
    JBrobe
    JProbe

    Before profiling you can also try to get couple of thread dumps and analyze them.

    Also you should watch app servers statistics:
    - JDBC connection pool
    - EJB pools and caches
    - HTTP request pool

    Typical issues:
    - you create 1 million ORM objects (entity EJBs) for your SQL result
    - you do not kill state full session beans after session time out


    But if you really want to understand, what happens in your app server, then you have to go for profiling ...


    Be aware that the profiling slows your app server so if your DB is slow under full load, but fast under profiling, you will not find it ...

    DF
  5. also set the xmx and xms to same values to avoid changing of the heap size

    >> dual CPU
    do you mean 2 CPUs or hyper threading? do not use hyper threading !!!

    http://docs.sun.com/app/docs/doc/819-0084

    this is a performance guide for the SUN's app server 8 (btw free for production also)

    it is applicable for any J2EE app server

    it has about 100 pages, so you can see that the profiling is tricky ...
  6. 100% CPU utilization doesn't present a problem if the application delivers required throughput.

    We always set the number of threads of the web container of the application server in a number that allow us to get a max use of CPU of about 80-90%. If you get a 100% use of CPU, of course you must set a lower number of threads.

    On the other hand, i suggest you to use a tool such as Quest PerformaSure in order to see what it is wrong.


    Jose Ramon Huerga
    http://www.terra.es/personal/jrhuerga