Encouraging failure: How Amazon AWS promotes innovation and invention

Big businesses can no longer rely on a brand name, a long history of success, or established infrastructure to guarantee their competitive advantage. Startups that are building and delivering services mostly or entirely in the cloud are proving to be highly disruptive. Spotify, Dropbox, and Airbnb are examples of small firms that have managed to completely change their respective industries starting with little more than an idea. The availability of cloud based resources such as Amazon's various Web Services (AWS) offerings has given entrepreneurs the opportunity to compete on an entirely new level, with startups being capable of challenging even the largest organizations.

Where people would traditionally scale out their production infrastructure, we're also scaling out our developer infrastructure. 

Weston Jossey, TapJoy

"If you want to succeed, double your failure rate." This quote is properly attributed to former IBM chairman and CEO, Thomas J. Watson, but it's a theme that permeated throughout Andy Jassy's keynote address at the 2014 AWS Summit in San Francisco. The Senior VP for Amazon Web Services spoke optimistically about the ways that the cloud promotes innovation by lowering the downside of failed experimentation, essentially making it safer to fail, which has the effect of encouraging innovation through experiment. At the outset of his presentation, Jassy outlined a number of interrelated reasons enterprises are now eager to leverage cloud resources to stay competitive.

Thriving under a cloud of failure 

Enterprises and businesses can't afford to move slowly in the face of this technological revolution, but how agile can an organization be when the average length of time to obtain and set up a new server is typically between 10 and 18 weeks for the typical enterprise? Such a delay has a crushing effect on innovation. Engineers don't bother to invent because it takes too long to get the servers they would need to test an idea. In many organizations, the frustration and delay isn't worth the effort, not to mention the fact that such delays seriously damper enthusiasm. It's no wonder that smaller, more agile competitors who are willing to turn to cloud based technologies such as Amazon's EC2 images are so much more likely to get a new idea off the ground quickly.

Any software professional who prides him or herself on innovation and invention knows there are two critical factors for success. First, the inventor must be able to experiment iteratively and quickly. Second, they must not be berated and punished if and when they fail. Any inventor who is pushing the envelope is going to experience more failure than success. It's simply part of the experimentation process, just like Thomas Edison figuring out hundreds of ways not to create a filament for an incandescent light bulb. If software engineers' jobs are on the line because they fear being called to account for wasted resources every time an experiment fails, this casts a pall over everything they do. Instead of trying anything truly innovative, they will only creep forward with incremental improvements on existing ideas. "In the cloud, things are radically different," said Amazon's Andy Jassy. "You can spin up thousands of instances or servers in minutes. If your experiments don't work, you can either give the resources back to us and stop paying for them, or you can reuse them for other experiments." 

Lower experimentation costs equals greater innovation

Weston Jossey, an engineer at TapJoy explains how his employer is taking advantage of excess resources from AWS in order to reduce development costs even further, making experimentation a very low cost activity. "One of the things we're able to do is really speed up our development lifecycle by using spot instances. Where people would traditionally scale out their production infrastructure, we're also scaling out our developer infrastructure. We're going as wide as we possibly can, running on these super-cheap spot instances. That's allowed us to reduce our development costs by $500,000."

In the cloud, things are radically different. 

AWS Senior VP
Andy Jassy

This freedom from high prices for infrastructure and compute resources has additional benefits that go far beyond the traditional group of inventors. The generation of creative ideas is no longer confined to R&D, senior management, or a software development team. Everyone in the company can begin contributing. The talent pool is now company-wide. "Now you're in a world where your employees know that there's a possibility that they may be able to deploy more quickly using the cloud. All of a sudden, you get a movement from learned helplessness to creative involvement as your employees begin to think about how to improve the customer experience." When there's a real chance that management won't turn down a bright idea due to cost constraints, workers have a greater incentive to think about the big picture, and in the end, that leads to innovation and invention.

How are you using AWS to accelerate innovation? Let us know.