The Five Sins of Mobile Testing

Discussions

News: The Five Sins of Mobile Testing

  1. The Five Sins of Mobile Testing (1 messages)

    The demand for mobile devices in all aspects of everyday life is growing exponentially. Whether your organization is adopting "bring your own device" (BYOD) and is rolling out a new internal mobile application or distributing mobile devices to drive employee efficiency, organizations need to meet the on-demand need for mobile apps, websites, and services. The job of IT is now to fulfill its traditional requirements and to meet the demand of mobile growth, without additional resources.

    Many enterprises have focused on monetizing and driving revenue by expanding what they do on the web to include mobile. But, some businesses are not prepared to expand to mobile. At times, upper management may make the executive decision to expand its mobile offerings without considering the time and manpower constraints that developers are limited to. Tight deadlines drive IT teams to rush through the development process of mobile apps and websites, leaving greater room for error.

    The growth of mobile has presented a new wave of complexity to development because of the increasing device diversity and OS fragmentation. Choosing the best platform to develop your mobile app has become increasingly complicated. The complexities in mobile continue to grow and create new hurdles for selecting and rolling out the right mobile strategies on the right devices. Fragmentation has created the need to build several versions of the same app or mobile website to conform to its variations (like size of the screen, screen resolution, OS, firmware, etc.).

    To help the process along, there are some key best practices that can be followed in order to develop and produce a successful mobile app that's user friendly and reliable. Simply avoiding these five sins can save valuable time, resources, and strain while developing a finished product that's functional and reliable, allowing you to bring your mobile strategy to market with confidence and assurance.

    Sin #1: Relying Solely on Emulation Software
    In the world of mobile, there are multiple platforms, OS versions, form factors, and carrier/ manufacturer customizations. It can be difficult to obtain all of these to properly create scripts and test for bugs or malfunctions. Developers often turn to emulated devices as an answer to their problems.

    While viable, especially in time and resource crunches, testing teams must take caution and understand that even with the most up-to-date software, certain quirks of mobile devices cannot always be replicated. This leaves room for error, especially while testing the behavior of an app on multiple devices. While emulation software can be helpful for basic functional testing, such as screen resolution, it falls short when trying to test one’s app or website over a real carrier network.

    The best solution to yield more accurate test results before pushing your app to market is to test on real devices. The hybrid approach of using the cloud to run scripts through real, connected devices may best suit developers who need to test against a multitude of handsets but just don't have physical access to them. Using a service that gives remote access to real devices allows developers to receive more accurate test results than using emulated software. Interactive manual testing of consumer-facing mobile apps can provide an easy, cost-efficient way to test functionality, usability and performance for both native and web-based mobile apps.

    Sin #2: Using Jail Broken Devices
    Unlike the Android OS, Apple's iOS uses a strictly closed operating system. This adds a major roadblock for testers needing access to objects from an app. Many companies adopt jail breaking with trepidation because of the potential legal ramifications with manufacturers.

    While the Digital Millennium Copyright Act (DMCA) temporarily allows jail breaking (rooting) of mobile phones, the exemption does not carry over to tablets. The advent of tablets and the legality issues of jail breaking have made it increasingly difficult for testers to extract object recognition data from these devices. Fortunately, there are solutions available where you can tap into real devices on the cloud that can give you advanced and pixel perfect experiences.

    When you jail break a device, limitations to testing arise because you have altered the device by deploying an “agent” to control the device’s functionality and touchscreen capabilities. The developer no longer gets the truest experience on how that device would function, opening doors for error. True hardware integration is the most accurate way to test for mobile.

    Sin #3: Not Leveraging Automation
    While organizations are quickly adding support for mobile, balancing the ratio of effectiveness to efficiency in testing can be daunting.

    The cost of producing and testing so many apps can balloon mainly due to device fragmentation. Additionally, building many versions of a mobile app is a strain on resources and time-to-market. Businesses can see this as cost prohibitive to enter the mobile market and are forced to weigh the pros and cons of such an undertaking. However, there are automated mobile testing platforms and tools that provide efficiency and drive down costs through automated testing.

    Automated mobile testing can save testers time and increase efficiency by ensuring scripts are repeated correctly, resulting in less error.. The process of scripting, testing, verifying, debugging, back to testing, then verifying again can become overwhelming. Mix in the need to test over the spread of several platforms and devices, and the need for automation becomes a no-brainer.

    Automation is especially helpful when running the same scripts over multiple devices. Saving time and increasing quality are the biggest benefits of test automation. Automation can be the equivalent of a team of testers working non-stop, around the clock, running scripts on multiple devices, but faster and with less room for human error.

    Sin #4: Ignoring the Need to Support Internal Apps
    As more companies make an effort to streamline and make their employees more efficient with the BYOD movement, tension has increased for IT departments trying to support devices that are outside the infrastructure and security of the company. Mobile devices often encounter both security and performance issues when integrating a mix of personal and corporate applications, which were not built to co-exist and can expose one's network to malware attacks.

    BYOD is a growing reality for the enterprise. Employees want the convenience of using their own mobile devices for both personal and work related functions and organizations are becoming more open to the idea of letting employees use personal devices for work, leaving the compatibility issues in the hands of IT.

    The primary concern for enterprise IT and the BYOD culture is ensuring that enterprise apps are functional across multiple platforms and can support the needs of the organization’s users, especially when the applications are tied directly to functions that affect the business’s bottom line. It is critical to be prepared to test and debug at the drop of a hat, possibly over multiple devices. Creating a process of testing your mobile app at all stages of development is essential and will allow developers and QA teams to protect their networks while supporting their employee workforce.

    Sin #5: Skipping Testing on Real Devices
    Speed is of the utmost importance in app development. The faster you can develop and make your app available, the faster you can generate revenue from the app. This leads many development or QA teams to spend little or no time testing detailed functionality of their app or website.

    With the main focus on pushing out apps, organizations are taking risks by rushing through or skipping the testing process altogether. Apps like Google’s iOS Gmail app were not tested, failed, and were removed from the app store. Such failures could have been catastrophic if it had been a less-well-known company. Without proper testing, the reliability of the apps and services that you offer diminishes, and the future of your development will suffer. Testing is critical to any mobile website or app launch.

    For those publishing their app for an app store, releasing a product with too many errors may lead to a bad user experience and may affect the success of the app. With millions of mobile apps in stores, users depend on other user reviews to determine whether one is worthy of a download. Piling up negative user reviews from bugs and other incompatibilities can be a brick wall for the success of an app and can affect future revenue growth for the company.

    In the 2012 -2013 World Quality Report by Capgemini, hundreds of developers across several industries were asked if they had a proper standard with mobile app testing before they pushed their app to market. Only 20 to 30 percent of developers said yes, unveiling a clear need for proper testing in the market.

    The same report also states that the majority of organizations across many industries expect testing budgets to rise between now and 2015 as they realize more attention must be made to not just shift to mobile but to do so with quality results. Starting by avoiding common mistakes, teams can be well on their way to creating efficient applications for use by any target audience.

    How do you know if you’ve done it right? If you hear nothing from employees or through comments in the app store, you have successfully tested. Most of the time, the only time you’ll get feedback is when the experience is negative. Still, it seems that many businesses are taking their chances and pushing apps out to market. How risky are you? 

     

  2. I was looking for an informative article around mobile testing. Instead, I found this article which in my humble opinion is garbage. Calling "sins" to the items outlined made no sense to me or to the developers in my team. Especially, when your employer Mr. Galde, Keynote Systems, still offers jail-broken devices (sin # 2) as part of their DeviceAnywhere offering.  I will be looking forward to informative, insightful postings from you and your company in the future - this one, not so much.