TestMaker 4.3 is now available for immediate download. TestMaker is a framework and utility for building black-box and grey-box tests of Web apps, Web Services and SOA for functionality, scalability, and performance. The new version focuses on building jUnit TestCases that are used for functional and load tests.

Follow these links for details:

Download (21.4 Mbyte) http://downloads.pushtotest.com/TestMaker43.zip

Feature description http://www.pushtotest.com/Downloads/features.html

Release notes http://www.pushtotest.com/Docs/vreleasenotes

What's new?

TestMaker 4.3 replaces the previous text editor with J, a powerful text editor. J features Python syntax highlighting, easy navigation among method definitions, search/replace, advanced search using regular expressions, and code indentation.

A new Quick Start Helper window opens upon start-up to offer immediate choices to create a new test agent script, read documentation, and learn about the included sample test agents.

TestMaker 4.3 is distributed under a GPL General License. Previous versions of TestMaker are distributed under an Apache-style license. The GPL license is free and allows you to distribute and modify TestMaker, however, unlike the Apache-style license changes to TestMaker must be contributed back to the TestMaker project.

-Frank

I've been evaluating it recently, but haven't had much time to check. I threw it at some ugly webpages (i.e., not valid XHTML) and it really seemed to choke on various inputs. I created a simple valid page and it still didn't seem to send info. The agents don't seem to record, if that's what they're supposed to do. I'm not sure because the documentation, though seemingly plentiful, doesn't even give me an overview of what's expected or common pitfalls. Is the book (that was panned here not too long ago) a good starting point?

I read frustrution between your words. That's not good and I'm wanting to help.

For an overview of TestMaker and a guide on how to get started, I recommend you read the User Guide that starts here:
http://docs.pushtotest.com/tm4testmakerdocs/index.html

TestMaker is appropriate for building unit tests that may be run as load and scalability tests and service monitors. TestMaker is multi-protocol and supports HTTP, HTTPS, SOAP, XML-RPC, and the email protocols. For HTTP/HTML applications the New Agent Wizard uses a proxy recorder to watch your use of a browser and to write a Jython test script automatically. (For me agent and script are synonyms.) The resulting test script implements a jUnit TestCase and may be run immediately to test the functionality of the Web app. The same TestCase may be plugged into the XSTest framework - included with TM - to run multiple times concurrently as a load test.

I'm not sure what went wrong with the bad HTML you tried. The users at lists dot pushtotest dot com list is a good place to post the problem and get an answer. TestMaker is updated about every 6-8 weeks so your posting will likely have an immediate impact on the next release. Give it a try.

The rest of the documentation library index is here:
http://www.pushtotest.com/Docs/doclibrary.html

Hope this helps.

-Frank

I'm having problems to, tried to record a login to www.gmail.com

It resulted in a null pointer exception in the 'text output' area (before any result was returned to the browser – didn't get to the login part). Not sure what or where the problem is, as the console writes noting and I can't find any log files.

I've tried other url's, where it seems to work, so its not a browser setting problem.

- Morten

Two things might be going wrong with GMail and Google.com:

1) TestMaker's recorder does not support HTTPS. By the time the recorder's proxy receives the request from the browser the request is encrypted. There's been much discussion on alternative architectures but nothing has been decided yet.

Unfortunately it seems that when the recorder is given an HTTPS request it doesn't fail gracefully but instead throws the null pointer exception. Graceful handling of the error is on the list of changes for a maintenence release.

2) Google's terms-of-use policy says that they do not allow their services to be recorded. I've noticed the Google search service returning a not-allowed exception to TestMaker where the same request through IE works.

Hope this helps.

-Frank

1) TestMaker's recorder does not support HTTPS.

If it can't support recoding a web site behind HTTPS how useful will it prove to be? How many business applications wont be HTTPS?

Cynergy Systems

The first STAR (Software Test Automation) conference I attended lead off with a key note address of "Scripting Is Dead." The speaker described record/playback as a one-direction serial process that required manual changes to an existing script when the host service under test changes. When you need to do maintenence on a script you might as well be coding, and if you're coding then you should use a test development framework to facilitate maintenence instead of a record/playbaack system.

Through TestMaker I urge QA technicians who do not code to become QA engineers that write code. The recorder is an assistance to help them learn coding as a skill. It outputs a jUnit TestCase from which they can begin writing the code needed to prepare a request and validate a response.

Within TestMaker is the Jython scripting language, HTTP and XML parsing libraries, an extensible protocol handler library, a framework for running TestCases as load tests, and a charting library. These are the components of a framework that any developer assembles to take a framework approach to testing.

-Frank

TestMaker runs tests using HTTPS but the recorder doesn't create test scripts from HTTPS requests. In my experience, many corporate developers have HTTP access to HTTPS services (either permanently or only while a test is being written.) They can record a script, change the HTTP URLs to HTTPS and then run the script.

HTTPS is a problem for many of the testing tools (JMeter, The Grinder, MaxQ, etc.) There's been a lot of discussion about how to overcome this. In general I've seen two proposals put forward that are workable:

1) Recorder with an SSL certificate. In this case the recorder has its own server-side SSL certificate. The browser makes the request and gets the certificate from the recorder. The big problem here is that server-side certificates are usually bound to an IP addres of a server so it would have to be dynamically created by the test utility. And if the certificate isn't Verisign, Thawte, or one of the other certificates the browser recognizes then the browser displays an ugly "Cannot Validate Certificate" error. (Don't get me going about the monopoly power of the certificate vendors. agh!)

2) The recorder hooks into a browser. In this case the recorder sits in a browser like Mozilla and watches the user on a host. The downside here is mintainence as the browser changes.

I like #2 better.

Anyone have feedback or want to suggest #3?

-Frank

1) Recorder with an SSL certificate. In this case the recorder has its own server-side SSL certificate. The browser makes the request and gets the certificate from the recorder. The big problem here is that server-side certificates are usually bound to an IP addres of a server so it would have to be dynamically created by the test utility. And if the certificate isn't Verisign, Thawte, or one of the other certificates the browser recognizes then the browser displays an ugly "Cannot Validate Certificate" error. (Don't get me going about the monopoly power of the certificate vendors. agh!)

I have quite a bit of experience with this subject having been an engineering manager for a product which did screen scraping of web sites. Our scraping engine was held in a J2EE web app. We could capture from HTTPS secured sources. As you state, we placed a server cert inside the web server. But contrary to what you said, it doesnt have to be an expensive paid for cert with the host IP in it. You can simply generate any old cert from the keytool, install it into your server, and viola. Yeah, the browser will complain about it, but simply check he box that says "Always accept" and ta-da. Problem solved.

Scripting is horrible. The whole value has to come from the recording tool.

2) The recorder hooks into a browser. In this case the recorder sits in a browser like Mozilla and watches the user on a host. The downside here is mintainence as the browser changes.I like #2 better.Anyone have feedback or want to suggest #3?-Frank

Yes we did this solution as well. Either work.


Dave Wolf
Cynergy Systems

Thanks for posting your experience. Glad to hear it went well for you. It shows I shouldn't be afraid of the dialog. Thanks. -Frank

Is it possible to record the script on a non-https server, then convert the script easily to run against the same app on an https server? In other words, do the recording on a dev box without ssl enabled, then simply update the script to use the https protocol on execute against an https enabeled server?

Anyone know if this is possible? I can see why recording would be difficult with https in place. Maybe this is a possible solution?

Do most other tools like this (LoadRunner, JMeter) have the ability to record over https? That seems difficult to implement.

Yes, TestMaker will record a script on an HTTP server. The script is a Jython script that includes the URLs to be called. One can modify the URLs to point to the SSL enabled URLs and then playback the script.

JMeter has the same issue of not recording HTTPs. (See http://wiki.apache.org/jakarta-jmeter/JMeterFAQ#head-d7d6b439cbb64a17ac50c302cf9bbc483ccc0d3e for info.) However, the JMeter FAQ recommends using BadBoy (a $40 shareware utility for record/playback that includes HTTPS, info is at http://www.badboy.com.au/)

LoadRunner, WAPT, and e-Valid do HTTPS record/playback.

-Frank

Gmail makes heavy use of the XMLHTTPREQUEST. Does TestMaker support that?

I'm having problems to, tried to record a login to www.gmail.comIt resulted in a null pointer exception in the 'text output' area (before any result was returned to the browser – didn't get to the login part). Not sure what or where the problem is, as the console writes noting and I can't find any log files.I've tried other url's, where it seems to work, so its not a browser setting problem.- Morten

I tried TestMaker then switched to TheGrinder and never looked back.

Hi,

When I record the Web applications with TestMaker 4.3 on Linux, it allways include several statement like

self.get( '''http://<my_hostname_here>/favicon.ico''')

What is this? It always fails when I replay the agent.

Thanks,
-Wenbo