Apache has announced the release of Apache JMeter 2.2, an application designed to load test functional behavior and measure performance. Originally targeted for web applications, it has expanded to cover other measurable components as well. Many bugs have been fixed, and some new capabilities have been added. JMeter works by acting as the "client side" of an application, and measures response time; it serves to generate load more than measure the impact of load on the server side. As such, it's one half of the testing arsenal; the other half consists of a tool to watch metrics on the server side, such as thread counts, CPU loads, resource usage, and memory usage. Most of the new capabilities won't be of much use to typical users, although some may prove to be handy, such as the new reporting features and the ability to override the HTTP handlers (and the ability to use other HTTP methods), but the list of bug fixes is pretty impressive. Apache has recommended that users upgrade to 2.2. What features would you like to see added to future versions of JMeter?

Things I'd like to see from JMeter: 1) HTTP session recording to produce load tests -- similar to what Maxq does, probably not using Python -- http://maxq.tigris.org/ 2) Better support for distributed testing, the current features are fairly underwhelming compared to stuff like SLAMD (http://www.slamd.com) or CLIF (http://clif.objectweb.org). 3) Some industry movement towards a way to describe HTTP sessions in a standard format. CLIF and Maxq both support ISAC (XML-based), but it's hard to consider that a standard at this point.

Things I'd like to see from JMeter:

1) HTTP session recording to produce load tests -- similar to what Maxq does, probably not using Python -- http://maxq.tigris.org/

JMeter already supports this, AFAIK.

Things I'd like to see from JMeter:

1) HTTP session recording to produce load tests -- similar to what Maxq does, probably not using Python -- http://maxq.tigris.org/

JMeter already supports this, AFAIK.

That is correct, it does support it already http://jakarta.apache.org/jmeter/usermanual/jmeter_proxy_step_by_step.pdf The documentation could use lots and lots of improvement, so it's understandable users have a hard time using all the features. I like to clarify the reporting GUI doesn't work yet. I started to work on it, but got side tracked by my day job and haven't had time to get back to it. There's basic XSLT templates for generating reports from JMeter logs. There's lots of areas where JMeter can improve, but the problem is finding time to work on it. My bias 2 cents, since I work on jmeter. peter

Sorry to see that I've missed the HTTP Proxy recording in the past, this obviously solves at least part of what I'd like to see from JMeter. Right now the main shortcoming, which is present in other open source recording tools as well, is that it can't handle SSL. I understand the technical reasons for this limitation, but it makes it very difficult to use the tool in the kind of work I do. James

Many users simply use badboy to record HTTPS and then use JMeter to run the test. It's quite affordable compared to paying Mercury big bucks. Not every company is willing to put out huge sums of money for mercury. peter

Many users simply use badboy to record HTTPS and then use JMeter to run the test. It's quite affordable compared to paying Mercury big bucks. Not every company is willing to put out huge sums of money for mercury.

peter

Unfortunate for JMeter (and for those of us trying improve investor returns), not every company is willing to stop putting out huge sums of money for Mercury. But even if you can't load test an application as a pre-production activity because the infrastructure folks can only say the M word, I've found JMeter to be really useful during integration testing, not only for the app, but for database query performance. It's a nice feeling talking with Mercury techs when you already know what the results are going to be.

It's a pity for it is basically 2 lines in the init code of the application to allow SSL support in java.net.URL. System.setProperty("java.protocol.handler.pkgs","com.sun.net.ssl.internal.www.protocol"); Security.addProvider(new com.sun.net.ssl.internal.ssl.Provider()); (please see http://www.javaworld.com/javaworld/javatips/jw-javatip96.html for details)

It's a pity for it is basically 2 lines in the init code of the application to allow SSL support in java.net.URL.

System.setProperty("java.protocol.handler.pkgs","com.sun.net.ssl.internal.www.protocol");
Security.addProvider(new com.sun.net.ssl.internal.ssl.Provider());

(please see http://www.javaworld.com/javaworld/javatips/jw-javatip96.html for details)

Well, of course JMeter can load-test using SSL/HTTPS connections. What JMeter doesn't do is record your browser activities while using SSL/HTTPS.

It's a pity for it is basically 2 lines in the init code of the application to allow SSL support in java.net.URL. System.setProperty("java.protocol.handler.pkgs","com.sun.net.ssl.internal.www.protocol");
Security.addProvider(new com.sun.net.ssl.internal.ssl.Provider()); (please see http://www.javaworld.com/javaworld/javatips/jw-javatip96.html for details)

JMeter could "in theory" record secure traffic, but it's not a trivial change. The HTTP sampler already handles the HTTPS correctly. What the proxy recording feature doesn't do currently is have a checkbox that says, "use HTTPS". The user would need to enter the non HTTPS URL in the browser like "http://www.domain.com" and JMeter's proxy needs to convert it to HTTPS request. This way, the proxy can record the traffic unencrypted and send the request to the server with HTTPS. It's not that hard to do, we just haven't had time to work on it. peter

OK, that's very clear. Thanks for the clarification. design4speed

JMeter has full swing and lighweight component support. Does that mean, we can use it for load testing of Applets as well? -Nikhil.

JMeter has full swing and lighweight component support. Does that mean, we can use it for load testing of Applets as well? -Nikhil.

If you mean use JMeter to test a GUI, the answer is no. JMeter runs in a GUI or console mode. The main benefit of JMeter is that it is pluggable and users can write their own plugins. For example, I wrote a JUnit test plugin for JMeter so that users can reuse existing JUnit tests. Another useful feature is reading standard HTTP logs and generating reports from them to perform simulation testing. I wrote that plugin for situations where an user wants to run the same exact load on 2 systems and compare the performance. Over the last few years, I've tried to improve the docs, but honestly writing good or great docs is very hard. Hopefully the current docs are decent. peter

One of the big flaw of all the load testing software (including the expensive one AFAIK) is that they do not emulate javascript execution like the browser. In recording proxy, it is compensated for links/resource load like images, etc. because the proxy will grasp all URLs emitted by the browser. But the usage of javascript becoming more and more massive (AJAX, complex interaction / computation done on client side) that I am not sure it will be long possible to skip execution of js in the load injectors like JMeter. Is there any plan to support that in the future of JMeter, for instance using Rhino and a very limited Java object environment emulating browser's Window, Document, Frame, etc. objects in js ?

One of the big flaw of all the load testing software (including the expensive one AFAIK) is that they do not emulate javascript execution like the browser. In recording proxy, it is compensated for links/resource load like images, etc. because the proxy will grasp all URLs emitted by the browser. But the usage of javascript becoming more and more massive (AJAX, complex interaction / computation done on client side) that I am not sure it will be long possible to skip execution of js in the load injectors like JMeter. Is there any plan to support that in the future of JMeter, for instance using Rhino and a very limited Java object environment emulating browser's Window, Document, Frame, etc. objects in js ?

several people including myself have toyed with the idea of embedding a full web browser in JMeter. The hard part is finding a good browser written in Java that is easily embedded. the other issue is whether the code is apache license compatable. this request comes up frequently, but the challenging of doing it is rather difficult. From a coding perspective, it probably isn't all that hard, just time consuming. peter

One of the big flaw of all the load testing software (including the expensive one AFAIK) is that they do not emulate javascript execution like the browser. In recording proxy, it is compensated for links/resource load like images, etc. because the proxy will grasp all URLs emitted by the browser. But the usage of javascript becoming more and more massive (AJAX, complex interaction / computation done on client side) that I am not sure it will be long possible to skip execution of js in the load injectors like JMeter.

Is there any plan to support that in the future of JMeter, for instance using Rhino and a very limited Java object environment emulating browser's Window, Document, Frame, etc. objects in js ?

Honestly, this isn't as critical as it appears at first glance. Any sufficiently advanced recording mechanism will still record all of the fetching that JavaScript does during the test creation. Those fetches can then be directly included in the test plan without needing to actually execute the JavaScript. Yes, there are scenarios this doesn't cover, but it has been good enough for all of the complex AJAX sites I've worked with to date, with the notable exception of anything that has to pass over HTTPS. I'm looking at BadBoy now and it may solve that problem quite nicely. It's not open source, but it's definitely cheap. James

Honestly, this isn't as critical as it appears at first glance. Any sufficiently advanced recording mechanism will still record all of the fetching that JavaScript does during the test creation. Those fetches can then be directly included in the test plan without needing to actually execute the JavaScript.

Yes, there are scenarios this doesn't cover, but it has been good enough for all of the complex AJAX sites I've worked with to date, with the notable exception of anything that has to pass over HTTPS. I'm looking at BadBoy now and it may solve that problem quite nicely. It's not open source, but it's definitely cheap.

James

You are right, in most case it is enough. The big issue usually is once you have recorded one scenario with a proxy, you have to vary smartly the parameters to compensate cache effect and simulate user variability. In this case, you might want to choose among different links in a group, change parameters in URL, etc. This might be easy if the URL encoding is straightforward, but it can be also very challenging if you are using complex encoding. In such cases, simulating browser events is much easier from a test coding perspective, and closer to reality. But I acknowledge this is a rather extreme case. design4speed

Honestly, this isn't as critical as it appears at first glance. Any sufficiently advanced recording mechanism will still record all of the fetching that JavaScript does during the test creation. Those fetches can then be directly included in the test plan without needing to actually execute the JavaScript. Yes, there are scenarios this doesn't cover, but it has been good enough for all of the complex AJAX sites I've worked with to date, with the notable exception of anything that has to pass over HTTPS. I'm looking at BadBoy now and it may solve that problem quite nicely. It's not open source, but it's definitely cheap. James

You are right, in most case it is enough. The big issue usually is once you have recorded one scenario with a proxy, you have to vary smartly the parameters to compensate cache effect and simulate user variability. In this case, you might want to choose among different links in a group, change parameters in URL, etc. This might be easy if the URL encoding is straightforward, but it can be also very challenging if you are using complex encoding. In such cases, simulating browser events is much easier from a test coding perspective, and closer to reality. But I acknowledge this is a rather extreme case. design4speed

There are some cases, like .NET 2.0 viewstate, which require work arounds. As long as a website doesn't do funky proprietary things to main state in the browser, JMeter should work. peter

You might want to look at JDIC web browser. You can embed either IE or FireFox but it uses heavyweight components. Hope That Helps, -Tony

You might want to look at JDIC web browser. You can embed either IE or FireFox but it uses heavyweight components. Hope That Helps,
-Tony

IE would only work on windows, so it's not really a cross platform option. Firefox isn't written in Java, so lots of work would be needed to embed it. The cost of doing something like that affect the over all performance of JMeter. Starting up a full blown browser or multiple instances of a browser for stress testing wouldn't work well. For functional testing, it would be good, but the performance trade off isn't trivial. JMeter wasn't designed to test client side stuff, so that's not a feature or supported. The real benefit of embedding a browser is to record complex user behavior easier. running a test using an embedded browser probably doesn't make much sense. atleast not to me, but that my bias opinion. the proxy approach is how many other stress testing tools handle recording test plans. it's a good model and works well. peter

Does anyone know where can I find the release note for 2.2 version. I have looked in the JMeter apache site but did not find anything there related to this version. Thanks