A friend gave me a problem to build a painless functional testing tool. They have all kinds of testing tools [Rational, Mercury, Agitor (Unit testing) ]. Still they face following difficulties:
- After sometime the recorded scripts become unmanagable
- Too much to record for a good coverage
- Skill ncessary to operate the testing tool and writting the scripts
-Too much setup work and regular registance from developers
-Data changes invalidate the recordings.
OK, if you are thinking being discipline will make it happen; you are right. But it pains :( Let the machine be deciplined and do the heavy lifting.
I took this problem and applied the search engine to solve it. First,
Preparing the request
- Record from all kinds of clients which access a server.
- Sense and create simillar requests
- Minimal human touch
I will write TE for test engine. The TE server will listen for requests from the client. After accepting a request, it establishes a new connection to the application server. Now TE works as a conduit between application client and application server. Intermediately it records all the communications. Putting the TE is easy by manipulating the hosts file or the web prowser proxy.
Consider our application is a web application. The browser client makes an GET or POST request to the application server. These requests are pretty clean. However, sometimes they carry some past burden as
- The session information and things like that.
- Some last processing details
- Cookie information
TE need to undersatnd and know all these information. TE will do it by passing the request through many filters. Here I am proposing few filters to detect what the request parameters are:
- The recorded cookie
- The last results returned; If any matches we know the source. (Example session ID)
- Coded entry in the forms of the application. Ex. Instead of real employee write empName. This is a instruction to TE to replace it with a real employee accessing other data sources like template database, test database, user provided values, regular expression based random values... Other examples will include phone 888-8888888.
- Direct specification of tablename.fieldname
- Matching sample data type from the provided test data search index.
Working on the response result
- Sense and perform first phase of validation.
- Picking only gold from the result
- Minimal human touch with sampling
TE generates large amount of similar requests and plays automatically. TE catches server responses. Most of the content are JUNK to us. TO remove the JUNK it needs to go through the filters of:
- Parse the content of xml, html, swf files.
- Throw the unnecessary words. What is left is gold.
Try to sense the results, This will help system to validate in absense of the test engineer.
Now all these results are presented to the test engineer and test enginner validates the output. Test engineer validates the sensed fields. On need basis test enginner maps the result field to D/B table fields. Now for next play on the same request, TE knows what are the outputs to perform first phase of validation.
TE provides configured 5% record & play dialogue for validation.
Are you thinking why I need a search engine? If yes, I have put this to boost up the performance. Once the test data table goes to single search engine index; All tables including fields are searchable to enhance the sensing mechanism.
Why You think this "will not work"?
Posted with architecture diagram at VOX