September 3, 2004

Frank talks about fear and how it can derail efforts to find and solve scalability and performance problems.I've been around a lot of software development efforts over the years and one component of these projects has remained constant: fear. Software development is really, really hard. There are going to be times when it seems impossible to get from here to success. These are the times that fear can grip an engineer to the point of inaction. This is especially true for QA technicians needing to understand and solve scalability and performance problems at the end of a software development effort. Many times fear keeps the QA tech from testing.

I see fear-driven behavior a lot when companies engage PushToTest services. Consider the example of one recent customer. The company decided to provide its users with a desktop application to provide services to its managers. This was a good use of the PushToTest TestMaker environment because we could easily create intelligent test agents that make requests through the desktop application to their information system and learn its ability to serve large populations of users. The test agents drive the system like real users. This puts the test agents at the start of a very long line of information services that are required to answer the requests.

Consider how the service responds to the typical user's behavior. The test agent begins by logging-in, then doing several queries, then creating an appointment with a manager, and finally by logging-out. The information system that provides the portal service uses an LDAP server to authenticate users. The service then uses PeopleSoft for the business logic of the application, and Oracle on the backend as a database. The desktop application makes SOAP-based Web Service requests to a middle-tier application that makes the calls to LDAP, PeopleSoft, and Oracle. It looks something like this:

I often find myself watching a PushToTest customer's QA technician listing out the reasons why not to proceed with a test of a system.

1. When I ran the test agent it encountered an error from the service, so I stopped the test.
2. The last time I ran a scalability test it looked like the response times were way too long, so I stopped the test.
3. I don't think the test agents are doing the right thing, so I didn't run the test.
4. I noticed the test was making the server go crazy, so I stopped the test.

In each of these cases, had the test run it would have provided information about the system under test. The telling sign that the QA technician is experiencing fear is "so I stopped the test." The best way I have found to counteract fear is to understand that actionable knowledge comes from test experience. The testing view of the world is that errors, slow response times, wrong functions, and servers going crazy are good things to find.

After all, it is better for the test agents to experience failure, they don't mind. Real users will.

About the author