Performance and scalability: Looking for an open source solution
- Posted by: Mandeep Khambay
- Posted on: October 22 2001 19:44 EDT
I am looking to provide my client with an open source solution, right from web server, application server, to database server.
One of the key concerns for us is high number of hits on the site and ability to support high number of simultaneous users (like atleast 300-400)
I am looking for a very high-end app server solution.
Would JBoss be robust enough to handle high number of simultaneous users, hits?
any performance issues or suggestions for any other app server...?
In that regards, would it be comparable to Weblogic Enterprise Server?
We are planning to go for Apache, any concerns???
MySQL or PostgresSQL - would these database be able to support such high traffic, any issues???
I would really appreciate if someone be able to guide me on all of the above.
- Looking for an open source solution by Toby Hede on October 22 2001 21:02 EDT
- Looking for an open source solution by Mandeep Khambay on October 23 2001 10:31 EDT
- Looking for an open source solution by Pranab Ghosh on October 23 2001 15:03 EDT
- Looking for an open source solution by Lofi Dewanto on October 28 2001 18:10 EST
You really are limited to a few core products for Open Source Enterprise Java. This is a good thing, because it means that these technologies have a fairly large developer base and relatively high number of installations.
JBoss is more than capable of meeting the needs of a large number of App Server deployments, and is used in production by a number of clients.
For the type of loads you are talking about, it seems some type of clustering solution may be required, regardless of the app server chosen. It is arguable as to whether JBoss is comparable to Weblogic Enterprise Server and as I have not used Weblogic Enterprise I cannot comment. My suggestion is to roll out some typical cross-sections of your application architecture and do some performance and scalability testing.
Apache is fine, the most used webserver around. Remember that you will also need a Servlet engine, my suggestion would be a JBoss/Tomcat or JBoss/Jetty stack...you can download these preconfigured.
I would suggest PostgreSQL rather than MySQL, although I am a personal fan of Interbase too. There are a number of issues with MySQL in terms of the features it provides and I do not think it is of an Enterprise level...For your database system, the physical hardware becomes a really crucial issue...
Thanks. This is really good info.
Does latest version of JBoss have clustering support?
I have seen earlier threads that they would be introducing it.
For Servlet and JSP support, JBoss definitely relies on a Servlet engine.
Would Tomcat or Jetty be able to handle the high traffic??
I have tried to investigate this and have found suggestions that Jetty would scale up much better.
For physical hardware, would a PC machine running Linux, with around 1GB of RAM be sufficient as a database server, application server and web server (all 3 separate machines)
OR would a SUN system be required to support such high traffic...???
JBoss clustering is set for Version 3, which is likely to be released late this year. You can download a beta of V3 if you would like to get started sooner. I have the source for V3 and the feature is already there (the beauty of Open Source is that the developers can't say that a feature will be there when it isn't -> you can always see how they are progressing).
That said, there are scalable ways of achieving clustering without using inbuilt clustering features of servers themselves. You can route users to particular machines for the life of their HTTP session, using hardware or even a Servlet type routing system, not that hard to do really. See: http://www.onjava.com/pub/a/onjava/2001/09/18/jboss.html
Indeeed, you can even stear clear of Session information altogether....this way you are not dependent on users being hooked to particular App Server instances.
As to physical hardware, seperating your database, application server and webserver is the best way to go. But, the demands on the database tend to be much greater...this is consistently the bottleneck in my systems and I would suggest that a disproportionate amount of computing power should be given to the database. As well as plenty of RAM and a fast processor, your database needs a really fast disk system, preferably RAID, the particular level of RAID depends on your fault-tolerance demands. Advantage can be gained from clever partioning as well..running the OS of one disk, database application on another disk, data on a RAID array, and perhaps even the log on a RAID array. Databases tend to spend a fairly significant amount of type writing and reading to disk and work here can really speed things along.
The webserver can be a much smaller machine as a lot of its work will be in routing requests to the App Server and delivering static images and files to the browser client.
Currently I am building a J2EE based system with end to end open source technology including Apache, Tomcat, Struts, JBOSS and MySQL.
Could you please elaborate why you prefer Postgress over MySQL. Form my research, MySQL seemed to be better choice than Postgress.
The J2EE System that you are building, does it have a requirement for high traffic/hits.
If so, can you please give me some information on how you were able to select each of the technologies you have listed.
In my opinion, MySQL is not a relational database. There are certain features that are really crucial to a true relational system and MySQL does not support them.
For a start, make sure you have the transaction-enable version of MySQL...prior to Version 4, transactions were not supported in the core MySQL release...
The MySQL site states that future versions will support: 'nested subqueries, stored procedures, and foreign key integrity rules'.
Personally, I think that some of these things are rather essential for Enterprise applications.
MySQL is a good system and this is not a holy war. It just pays to be aware of its limitations and leverage its advantages -> it is incredibly fast for read-centric siutes, for example.
Anyway, do a search on Slashdot (http://www.slashdot.org/) for 'MySQL' to get more details.
At Slashdot (http://www.slashdot.org/) I found really good threads comparing the 2 databases.
If anyone is interested in the comparisons between MySQL and PostgreSQL:
I'm using 100% Open Source products for my production site:
http://www.openuss.org (OpenUSS itself is also Open Source ;-))
I'm using 2 servers:
- ServerI (Win NT, PII 300, 512MB)
- ServerII (Win 2000, PII 500, 1GB)
- Enhydra 3.1 as Servlet Runner (ServerI)
- JOnAS 2.3 as EJB Server (ServerII)
- InterBase 6.0.1 as Database Server (ServerII)
- James as SMTP Server (ServerII)
OpenUSS handles at the moment 1000 until 1200 users.
My experience with InterBase:
No problem, but you have to update the interclient.jar for JDK 1.3. I'm saving all the documents (blobs) in the database and it works great with InterBase.
My experience with Enhydra:
Enhydra has no problem at all to handle many users!
My experience with JOnAS:
For a while I got some problems with JOnAS... as if JOnAS will never free the memory. So everytime, JOnAs always "eats" the memory and never frees it. At the end JOnAS will crash with "no memory available". After optimizing some EJB codes (I'm using Servlet - SB - CMP EB), JOnAS works well:
- Using isModified.
- Using 1 GB memory for the ServerII (with HeapSize 900MB).
- For a long list never use Entity Beans. Instead use DAO (Session Beans use directly SQL statement)
I also use some optimizations for the JVM:
- Server optimized.
- Incremented GC.
Hope this helps!