Testing strategies and mobile monitoring are keys to the mobile ALM process
By Eran Kinsbruner, Perfecto Mobile
To take advantage of the huge business opportunities offered by mobile applications, enterprises need to build mobile testing strategies that address the extremely fragmented and dynamic nature of the mobile market. While applications must run perfectly and perform at the highest levels on new devices and new mobile operating systems, those same applications also need to continue to support older legacy devices. Automated functional and performance testing are critical for ensuring maximum testing coverage prior to an application's release. However, the process doesn’t end there.
To ensure that business-critical applications used in key industries such as banking, security, productivity and healthcare continue to serve the customer and perform well across all devices, networks and operating systems, enterprises also need to monitor application performance in production. In this context, it is important to understand that in terms of device resources and network traffic, mobile apps are far more complex than their web and desktop based counterparts. As a result, to gain meaningful insight into the user experience, mobile monitoring must be done on real devices with real users. But just imagine how difficult it would be to simulate a native iOS or Android app running on a remote device in a specific network in a particular geography without real access to the devices.
Monitoring as an early warning system
Mobile monitoring should be based on the constant execution of key application transactions across devices and networks. In the case of below par application performance or availability, real-time alerts need to be sent to the operations team. The monitoring solution should serve as an early warning system to the organization about a problem, allowing issues to be recognized and addressed before any end users actually know about it. Since different devices behave differently on different networks, on different operating systems and at different times a day, the only way to effectively detect performance issues is through real-time, real-device monitoring.
Mobile Monitoring Management Tips
To overcome some of the challenges of mobile application performance degradation, here are some useful tips for building your mobile monitoring strategy:
The growing complexity of mobile applications and user scenarios introduces a major challenge for test automation. Monitoring solutions usually trigger automated scripts which are built to examine key performance indicators (KPIs) on the apps running on real devices. Not only do testers need robust cross-device automation, they also require the ability to create scripts that support both visual objects and OS level objects.
Weeding out false negatives
Another critical challenge in mobile monitoring is the need to distinguish between false negatives and real availability or performance issues. There is nothing more frustrating to your operation center staff than false alerts. Mobile monitoring products must support device redundancy to assure that the alerts being sent to the network operations center are real issues and not device-related problems.
Furthermore, by monitoring performance in a structured and scheduled manner, developers can collect key performance indicators from the device under test and gain valuable insights into the end user experience. This means upgrades, enhancements and even the retirement of certain features can be done in a way that has the greatest, proven effect, rather than having such decisions made based on guesswork and speculation.
As noted above, mobile performance testing and monitoring is only effective when using real devices. Organizations should capture the real end user experience on real devices covering real scenarios, and gather metrics on key issues such as:
- Is the server being overloaded
- How congested is the network
- Specific mobile device and OS issues
This allows developers to address these performance and monitoring issues early on, enabling companies to avoid the threat of serious financial losses and damage to their reputation.
The diagram below illustrates a recommended mobile application lifecycle. There are various processes which can be used by an organization; each has its own value and benefits. Performance testing takes place prior to the release stage, while the mobile monitoring is conducted after deployment as the program runs in production.
Test automation across disparate devices
Using a cloud-based mobile testing environment, in which devices are fully governed, secured and always connected, helps to enable a mechanism for continuous quality, delivery and integration. Developers can create a build engine that can run at any time, as well as test on real devices anywhere in the world to get insights into the stability of the application, devices and service. Furthermore, developers and DevOps teams can test their device behavior on a specific carrier network while using a customized mobile operating system. This type of solution is also well-equipped to support test automation across different devices and platforms with support for visual and native objects. Using this type of hybrid approach streamlines performance testing and monitoring by enabling unattended testing. Moreover, the built-in scalability of a cloud-based monitoring solution addresses the need for device redundancy to ensure reliable alerts.
A fully managed cloud-based mobile testing solution frees application developers from the hassles of device management and governance. The mobile testing solution vendor can provide the technical expertise, enabling the development team to focus on the application without worrying about the infrastructure, devices or other technical aspects.
|Eran Kinsbruner is the director of product marketing at Perfecto Mobile, a leading cloud-based mobile application testing and automation company. Formerly CTO for mobile testing at Texas Instruments and a project manager at Matrix, Eran has been in testing since 1999 with experience that includes managing teams at Qulicke & Soffa, Sun Microsystems, General Electric, and NeuStar. The co-inventor of a test exclusion automated mechanism for mobile J2ME testing at Sun Microsystems, Eran has extensive experience in the mobile testing world. You can find Eran on Twitter @ek121268, LinkedIn, and his professional mobile testing blog at ek121268.wordpress.com. Eran also writes regularly for the Perfecto Mobile blog.|
How is your mobile monitoring strategy improving the customer experience. Let us know.
21 Jul 2013