- TestScenario is a new XML-based system to reuse unit tests as regression tests, functional tests, load tests, integration tests, and service monitors requiring no coding.
- Graphical environment to control tests and view results in real-time.
- Record/play-back unit tests of Web applications using the integrated point-and-click TestGen4Web utility.
- Create test suites of SOAP-based Web Services using the integrated Eviware soapUI utility.
- Create tests of AJAX and REST applications using the integrated HTMLUnit framework.
- Java 6 ScriptEngine (JSR 223) support enables you to write tests in any supported language: Java, Jython, Groovy, PHP, Ruby, and many others.
- Distributed test environment (TestNetwork/TestNodes) to vertically scale tests to thousands of users and horizontally scale tests to operate the test from multiple locations around your network
- Performance Comparison Utility shows changes in performance from one test to another.
- Monitors CPU, Network, Memory utilization as test operates.
- Transaction distribution analysis charts as test operates.
- Automatic archiving of test results in XML file format.
- Log to Relational Databases (RDBMS)
- Data Production Library (DPL) system to create dynamic data as test operates from XML files, Comma Separated Value (CSV) files, and Relational Database (RDBMS)
- Validate service responses using Data Production Library (DPL,) integrated XML parsers, regular expressions, and custom objects.
- Operate PushToTest As A Service (PAAS) by making a SOAP request to TestMaker to operate a test and return the statistics of the test results. Includes fully functional interoperability with SoftwareAG/WebMethods X-Registry repository.
- Start, pause, stop, and change the order of test case operation as the test operates.
- Service monitor panel shows service status dashboard.
- Notification of test operation by email, SMS, pagers.
- Expanded protocol handler support (HTTP, HTTPS, Apache SOAP, Apache Axis SOAP, REST, AJAX, email) and extensible to add your protocols.
- Backwardly compatible with TestMaker 4 tests.
- Call TestMaker from the command-line to use with Continuous Integration Environments.
- Professional documentation, tutorials, screencasts, and reference guides.
PushToTest TestMaker 5 is a platform for test automation, service governance, and business service management. For Java developers TestMaker is a framework for turning unit tests into functional tests, load and scalability tests, and service monitors automatically. Top-level features and enhancements in PushToTest TestMaker 5.0:
- Posted by: Frank Cohen
- Posted on: August 16 2007 15:00 EDT
- Google TechTalk on TestMaker available by Frank Cohen on August 16 2007 16:12 EDT
- Re: PushToTest TestMaker 5.0 Now Available by William Martinez on August 16 2007 16:34 EDT
- Congrats! by Ole Matzura on August 16 2007 17:50 EDT
- Re: PushToTest TestMaker 5.0 Now Available by Dmitriy Setrakyan on August 17 2007 02:32 EDT
- Unit Tests to Functional Tests by Geoff Lane on August 17 2007 09:21 EDT
- Testmaker for variable load test by jay jay on December 21 2011 13:32 EST
I gave a TechTalk at Google this week on TestMaker 5. I showed how TestMaker reuses unit tests as functional tests, load tests, and service monitors. Google recorded the TechTalk and makes the video available at: http://video.google.co.uk/videoplay?docid=-4427578441647658650 -Frank
It seems like a major improvement from version 4.x. The UI changed a bit, and the recorded is improved. But what I like most is the unit test reuse. I've seen several companies where the unit tests were done using script languages like ruby or python. Those I can reuse by simple creating the test scenario XML (kind of a mix between a config file and a test definition language). I simply put the reference to my unit test there, add a couple of lines to define the concurrent users (100), and hit run! It will start hitting my application with the unit test but now using 100 simulated users! From unit test to load test in a couple of lines. The drawback is you have to code the XML, but that is not much work after all, since the XML is easy to read and you can just hit a button and it will generate a template for you to tweak. Another thing I liked very much was the DPL. You can now make more challenging tests by controlling the data used in the unit tests. It is as easy as implementing an interface and declaring the new DPL class in the XML. You can ask for a sequential access to the data rows, a random, indexed! I saw it and I immediately though about doing tests where the data size and types changed following a behavior like a real life run (imagin running a test when the first hour you have small packages and the next one you have huge packages, simulating a peak of data!). Still need to play a little bit more. William.
Good points. The DPL is a Data Production Library. It's a simple interface design to an object that will give a unit test data at runtime. For instance, when I test the log-in page to a Web application the DPL provides the log-in id and password from a Comma Separated Value file or a relational DB (RDBMS) query. I can also use DPL provided data to validate response values. -Frank
Frank! Congratulations on the long-awaited release of your powerful framework. We are really proud to be part of your offering with soapUI and wish you all best with the release. kind regards, /Ole eviware.com
Thanks. soapUI is an excellent tool for working with Web services. I remember Tim Bray asking an audience of software developers 2-3 years ago if anyone knew how to work with WSDL? I wish I could have pointed to soapUI back then. soapUI makes building unit tests and test suites for SOAP services very easy. soapUI is graphical, powerful, supports many common WSDL/encoding/platforms patterns, and now these soapUI test suites plug into the distributed test environment in TestMaker. Nice! -Frank
Pretty cool product! I was really impressed to see how a unit test and a load test were created just from a few clicks in Firefox browser. Kind of opens your eyes on what web testing should be. Are the tests created only in jython, or can it be JUnits too? Best, Dmitriy GridGain - Grid Computing Made Easy
Hi Dmitriy: Thanks for the nice compliment. About 3 years ago we adopted Jython as a dynamic scripting language to build tests. Back then you could implement a unit test in TestMaker by writing a Java class or a Jython script. We bundled jUnit with TestMaker 4 and included a Proxy Recorder that watches your use of a browser or application and writes a jUnit TestCase class in Jython that will play-back your test. Since then we've seen an explosion of useful unit testing frameworks: HTTPUnit, HTMLUnit, TestNG, DBUnit, and more. Since TestMaker is 100% Java it was easy to bundle these frameworks and support them with our Recorders and Wizards. TestMaker 4 adopted Jython as its scripting language about 3 years ago. I've been a proponent of running dynamic scripting languages on the JVM since then. I saw first hand how QA technicians with little technical ability were able to write Python code in Jython and run it on the JVM. That helps Java overall. TestMaker 5 adopted the JSR 223 ScriptEngine API in Java 6. So now you have your choice of language to create the unit tests. TestMaker 5 supports Java, Jython, Groovy, PHP, Ruby, and the 15+ others supported by the ScriptEngine API. -Frank
How do you reuse Unit Tests as Functional, Integration and Load tests? It seems like those things usually have very different goals and implementations. Have you found the testing Rosetta Stone?
I don't think we found the testing Rosetta Stone, it's more like it found us! So many Java developers today have adopted unit test frameworks to check the operation of a method or an entire class that supporting unit testing with TestMaker was an obvious decision. The tougher thing was to address the variety of users that need a testing platform. Testing in Java is changing. For instance, an Aberdeen report on SOA and test automation asked the question "Who does the testing?" And 53% of the responses said: Developers do the testing. The problem here is most developers don't have the resources a QA group or IT group have. So you wind-up with a wall between developers, QA and IT managers. TestMaker attempts to knock down the wall by providing a framework to operate a string of unit tests as a functional test, then reuse the test as a load test, and then reuse the test as a service monitor. TestMaker uses a TestScenario to define functional tests, load tests, and service monitor tests. A TestScenario is an XML-formatted document. We went the XML direction to make it easy for QA and IT managers - who normally have few programming skills - to build tests without programming. If anything, the TestScenario is the Rosetta stone for these 3 groups. -Frank
In Testmaker, is there a way to generate load that goes up in steps. For example, 100 users for first 30 minutes of test, then 200 users for next 30 minutes of test, then 300 users for the last 30 minutes of test.