The Cloud: Battle of the Tech Titans

Amazon, Google, and Microsoft are going up against traditional infrastructure makers like IBM and HP as businesses move their most important work to cloud computing, profoundly changing how companies buy computer technology

By Ashlee Vance

Amazon.com's (AMZN) squat Seattle headquarters looks nothing like the country club affairs found in Silicon Valley. There are no free soft drinks or volleyball courts. The light fixtures hanging from the ceiling in the reception area aren't fixtures at all but rather collections of extension cords fitted with bulbs. The receptionists lack computerized systems for registering guests. They simply write down visitors' names on a piece of paper. Such is low-margin life in online retail, where Wal-Mart (WMT) stands at the ready, waiting to take away your extension cords.

Most people recognize this Amazon: Jeff Bezos's hyperproficient Borders-killer; one of the few dot-com initial public offerings that didn't end up a punch line; fount of millions of smiling cardboard boxes bearing everything from dildos to diapers. Sitting in a sparsely decorated employee cafeteria, Andy Jassy pitches a newer if equally thrifty side of the e-tailing giant. Although all shoppers are welcome, this Amazon, he explains, is for business customers and isn't well marked on the home page. It's called Amazon Web Services, or AWS. As senior vice-president, Jassy heads up AWS, which rents out computing power for pennies an hour. "This completely levels the playing field," Jassy boasts.

AWS makes it possible for anyone with an Internet connection and a credit card to access the same kind of world-class computing systems that Amazon uses to run its $34 billion-a-year retail operation. "This will be a very high-volume, relatively low-margin business," Jassy says, sipping his third Diet Coke of the morning. He says he'd like to curb his 12-can-a-day habit, but why bother? If ever there were a time to get hopped up at Amazon, it's now.

AWS is growing like crazy. Although he won't cite exact numbers, Jassy claims "hundreds of thousands of customers" already use the service, and analysts at UBS (UBS) estimate Amazon will do about $750 million of business on AWS this year. In fact, a whole generation of Internet companies couldn't exist without it. Netflix's (NFLX) movie-streaming empire runs on it; Zynga, the social gaming company, uses it to handle sudden spikes in usage. AWS has become such a fact of life for Silicon Valley startups that venture capitalists actually hand out Amazon gift cards to entrepreneurs. Keeping up with the demand requires frantic expansion: Each day, Jassy's operation adds enough computing muscle to power one whole Amazon.com circa 2000, when it was a $2.8 billion business.

The physical expansion of all that data takes place in Amazon's huge, specially designed buildings—the biggest can reach 700,000 square feet, or the equivalent of roughly 16 football fields. These interconnected facilities, scattered all over the world, are where AWS conducts its business: cloud computing. The "cloud" refers to the amorphous, out-of-sight, out-of-mind mess of computer tasks that happen on someone else's equipment. For the past five years or so the cloud has been hyped by companies to mean anything that happens on the Web, which is how "cloud computing" came to rival "social networking" in overuse.

Right now, the cloud is small: It represented about 5 percent of the $1.5 trillion in corporate information technology spending last year, according to industry data supplied by International Data Corp. and Gartner (IT). Yet the phenomenon of businesses moving their most important and innovative work into the cloud is real and is profoundly changing how companies buy computer technology.

One other thing about the cloud: It's turbulent. It's getting to be war up there.

The battle roughly breaks down to two sides. On one is Amazon and two other rising superpowers of the cloud, Microsoft (MSFT) and Google (GOOG). That's right, the company that once touted itself as "The World's Largest Bookstore" sits alongside a search company and PC software giant as a leader in new business technology. These three companies are hardly allies, of course, but each poses a threat to traditional infrastructure makers such as AT&T (T), EMC (EMC), Hewlett-Packard (HPQ), IBM (IBM), Oracle (ORCL), and Verizon (VZ).

Those giants collectively control the other 95 percent of the market and sit on vast cash reserves earned by steering the creation of data centers inside large corporations. They specialize in selling and servicing high-margin products and reap the consulting fees that arrive from helping companies manage their equipment. These global powerhouses might seem invulnerable, but they must take Amazon, Google, and Microsoft seriously. The cloud war is only partly about next quarter's market share; it's also a culture clash.

Amazon, Google, and Microsoft have attracted a generation of converts to their message that technology can be fun again. They are the cloudpeople. This species has a vision of freedom and excitement. They see a world of innovation, where the digital engine rooms of the economy transform from unwieldy assemblies of parts into smooth-running dynamos. The cloud, they believe, can unleash the same kind of creative ferment going on in consumer tech, where smartphones and tablets have captured people's imaginations.

"Things are downright Darwinian right now," says Mike Olson, the chief executive officer of Cloudera, a startup that specializes in data analytics software. "There hasn't been this type of Cambrian explosion in corporate technology in 20 years."

On a January evening in Mountain View, Calif., about 80 programmers cram into an unadorned conference room, helping themselves to tables full of pizza and Blue Moon beer. The crowd is mostly male, mid-40s. There are a lot of T-shirts, gray hair, and ponytails. These are rank-and-file members of the Silicon Valley Cloud Computing Group, and they're here to learn.

At around 6:50 p.m., the attendees drift to the rows of tables and chairs where they've set up their laptops for the evening. At 7 p.m. sharp, a young, skinny guy at the front of the room calls for attention. Sebastian Stadil, the 26-year-old founder of the club, declares the meeting in session and cedes the floor to the guest speaker, Shashank Tiwari, a local cloud developer. The topic of tonight's three-hour presentation: working with the "sloppy quorums," "gossip-based membership protocol," and other fine points of a database technology invented at Facebook called Cassandra.

Stadil runs these night classes in his spare time. He's a dedicated cloudperson with his own software startup and feels it's his duty to educate his peers about the coming era. The admission fee is $20 a session, which covers the pizzas and beer. (The conference room, in the offices of prominent Bay Area law firm Fenwick & West, he finagles for free.) Attendees come to network and hear experts opine on Voldemort, Hadoop, Puppet, Chef, and other playfully named types of cloudware. "When the meetings started two years ago, it was mostly people in their twenties," Stadil says.

Since then, as cloud technology has crept through corporate IT departments, the crowds have gotten older. Much of the work done by people who managed computer systems is getting automated, which is forcing managers and engineers to upgrade their skills. Also driving the urge to learn: New IT tasks tend to be more intellectually demanding and nuanced than those of previous eras. Companies that used to dig through six months of customer data to tease out sales trends—"what's the average selling price" and "who's the biggest customer"—now use tools built by Google and other Web specialists to plow through 10 years' worth or more. And they're asking such computer-intensive questions as: What do people like? Why do they like it? What's the optimal coupon to send to their smartphones as they shop for it in aisle 6 at ShopRite? As Stadil puts it, "Companies don't need the guy who is good with a shovel anymore. They need a guy who is good with a bulldozer."

After the evening's talk, some of the old-timers grouse that cloud computing is just a new name for an old idea. It's true that technology types have long used a little picture of a cloud as whiteboard shorthand for "the Internet," or "the ethereal place somewhere on the network where work gets done." What Stadil's classes are about, though, is something more specific, and that message got through loud and clear to the other attendees. Some of them almost sound like true cloudpeople. "This time things really are different," says Robert Harker, a veteran systems administrator.

What's different this time—as compared with the rise of the mainframe or the PC—is scale. As the consumer Web exploded, the global mass of computer data went supernova. This year, according to IDC, the world's digital universe will reach 1.2 zettabytes, or 1.2 quadrillion megabytes. If you take every word ever written in every language, it's about 20,000 times that.

To cope with the onslaught, Web companies have built dozens of megadata centers—often at $500 million a pop. Some of the players—and here we return to Amazon, Google, and Microsoft—pioneered many of the techniques needed to deal with hundreds of millions of people trying to access unprecedented volumes of information. These companies built software that spreads data across the globe, automated the command and control of tens of thousands of servers, and refined data-center designs to be more energy-efficient. They had to out-innovate the big guys and do it on the cheap because their customers were consumers who weren't locked in with fat contracts and dedicated account reps. The next logical step for the cloudpeople: Sell the capabilities of their new supersystems to customers who would pay—other businesses.

In the early 2000s "software as a service" companies such as Salesforce.com (CRM) pioneered the model of renting niche applications to customers by the month. Hosting companies also offered to run companies' software in their far-off computing centers for monthly and yearly fees. But it was the Web companies, led by Amazon, that went a step further and sold these types of services on a utility basis. A company could fire up hundreds or thousands of computers on the fly and turn them off when the work was done. Instead of monthly or yearly contracts, customers pay only for what they use in computing cycles, bandwidth, and storage. "Pay-by-the-drink pricing seemed natural to us," Jassy says.

The most obvious appeal of such cloud services, of course, is the potential to save money. Business buyers have gone through decades of technology transitions, tacking new hardware and software onto the old. The average corporate IT department has to deal with the dreaded 70/30 rule where they spend about 70 percent of their technology budgets just trying to keep this jumble of products running, leaving only 30 percent to chase new ideas. Chief information officers have to buy equipment by the ton to meet spikes in computing demand or prepare for disasters, and then watch as that gear sits idle most of the time. While companies struggle to deal with this mess, they've become overwhelmed by the influx of data in the Internet Age. The promise of the cloud is shoving the costs of dealing with all that off the bottom line.

The cloudpeople suggest a daring complement to the cost-cutting: Adopt the rapid-prototyping, beta-testing lifestyle of the new era. Relinquish control of your technology infrastructure—you don't need it anymore. Let employees toss out new services and see what sticks. Innovate with impunity. "In the cloud," says Tony Scott, CIO of Microsoft, "there's no penalty for guessing wrong."

"This stuff finally works," says Andy Bechtolsheim. "If you're a startup, you would never build a data center again." Bechtolsheim co-founded Sun Microsystems and built the company's first computer workstation, which itself set off a revolution of lower-cost systems for businesses in the 1980s. He also made billions cutting some of the first checks to Google and VMware (VMW), the big enterprise-software maker. As Bechtolsheim sees it, the cloud is a foregone conclusion for new companies such as his latest venture, Arista Networks. The only hardware of any significance on the premises of Arista are the high-speed switches it sells to companies building large-scale cloud computing systems. Everything else—e-mail, order entry, customer management, Web software—sits on someone else's hardware and is someone else's problem.

Or take the case of Etsy, an online marketplace for handmade goods. It rents hundreds of Amazon computers every night to analyze data from the 1 billion monthly views of its website. When Etsy's engineers get to work in the morning, they have a wealth of data showing what types of clothes, furniture, and jewelry appeal to what types of people. Etsy has used this information to create product recommendation systems that let people rank their interest in a series of products and then end up with a list of 100 or 200 they might like. Consumers can also grant Etsy permission to trawl through their Facebook accounts and find products friends might like as gifts. If, for example, Etsy sees a person "liking" something related to Michael Jackson, it will offer up some Michael Jackson dolls or vintage clothes. Jason Davis, the lead scientist at Etsy, loves that he's got his hands on technical tools that used to be available only to retailers the size of Gap (GPS) or Ikea. "This wouldn't have been possible five years ago," he says.

Five years ago that flexibility eluded big companies, too. For all their resources, large institutions tend not to encourage technological initiative. The classic chain of events: When someone in an IT department wants to try out a new idea, that person has to justify the expense in budget approvals and PowerPoint supplications, and then wait weeks, if lucky, for the gear to arrive and start working properly. Heaven forbid someone in marketing or sales wants to try out a new idea. That means more approvals, justifications, and delays.

It was this type of tech-hungry underling who drove the initial interest in the cloud. Unbeknownst to their bosses, software developers were running off to cloud services—in particular, Amazon's—to test out ideas, often using their own credit cards to pay for work. For the first time, cube dwellers with ambition had the freedom to tinker on serious computing systems without much overhead. When something clicked, the developers could then take it to their bosses.

Just such a thing happened at Northrop Grumman (NOC), where a research and development team used Amazon's cloud service to develop an advanced cybersecurity system. The team rented Amazon's computers for less than a day to train machine-learning algorithms on more than 1.3 million files. When Justin Kessler, an applied mathematician at Northrop Grumman Information Systems, showed the results of the project to his managers, they were struck by the low price tag. "It was eye-opening, and they were thrilled at how efficiently we solved the problem on someone else's infrastructure," he says.

Other global corporations such as Bechtel, Eli Lilly (LLY), and Pfizer (PFE) have experienced this ground-up push. So, too, has InterContinental Hotels Group, which manages 650,000 rooms at 4,500 hotels worldwide. IHG has started a project to revamp its technological lifeblood, its central reservation system. The current version relies on decades-old mainframe technology; the new one is coming to life on Amazon's cloud. Bryson Koehler, a senior vice-president at IHG, says the company considered keeping the software in-house, just on upgraded hardware. "But this is a six-year journey," he says. "The last thing I want to do is move over to current technology and then be outdated by the time we finish the project." Koehler expects that pooling all of the company's data into the cloud will let IHG deliver information more quickly to customers regardless of their location, and to perform more complex analysis of users' behavior.

Strapped city, state, and federal technology departments have been some of the organizations most enchanted by the quick turnarounds the cloud promises. Miami, for example, has built a service that monitors nonemergency 311 requests. Local residents can go to a website that pulls up a map of the city and places pins in every spot tied to a 311 complaint. Before the cloud, the city would have needed three months to develop a concept, buy new computing systems (including extras in case of a hurricane), get a team to install all the necessary software, and then build the service. "Within eight days we had a prototype," says Derrick Arias, the assistant director of IT for the City of Miami. "That blew everyone away internally."

Companies such as Starbucks (SBUX) and Kraft Foods (KFT) have been early backers of Microsoft's cloud services, which include online versions of Office and its business software platform, Azure. Starbucks uses Microsoft's cloud e-mail to keep its army of baristas up to date about sales and daily deals. When Kraft bought Cadbury last year, it too relied on Microsoft software to merge the two companies' technology systems. Kraft has also begun conducting more complex computing experiments on Azure. "I want to change to a model at Kraft where basically our core infrastructure is part of the Internet," says Kraft CIO Mark Dajani. He's looking to upgrade about 10 percent of Kraft's infrastructure per year and wishes he could move faster. "If I built Kraft from the ground up today," he says, echoing the feelings of many big company CIOs, "I would leverage the cloud even more."

All this chatter about handing over crucial data and processes to places such as Amazon clearly annoys executives at the technology giants. "This is our 100th year," says Ric Telford, IBM's vice-president for cloud services. "We have had new competitors, new niches, and new markets come and go." Telford's counterparts at Cisco (CSCO), EMC, HP, IBM, and the other incumbents share similar sentiments. The old-school tech giants don't reject the cloud. They're just anti-chaos. Their view, with some justification, is that conducting key parts of business elsewhere carries risks. The cloud is fine for experimentation around the edges, but better to keep the mission-critical functions in-house. They contend that big technological shifts should not be left to enterprise-computing amateurs such as Amazon and Google. (Microsoft, which has sold vast amounts of technology to corporations for decades, can hardly be called an amateur.) Big tech understands the nuances of doing business on a global scale and how to take care of the finance, health-care, and government customers concerned about an information free-for-all. As Telford puts it, "You can't just take a credit card and swipe it and be on our cloud."

The heavyweights are encouraging customers to buy virtualization software, which makes it possible to run more business applications on each individual server. This means customers get more bang for their buck when buying hardware. To keep these new systems running smoothly as a unified whole, customers may also purchase new management software. Add these parts together, and you end up with a nimbler, more efficient version of traditional IT infrastructure known as the private cloud.

The private cloud is everything the incumbents love. They view it as a pragmatic transition toward a smooth-running computing infrastructure. Most customers will stick with the sellers they trust, paying for some consulting service to help them put the new technology to use.

The incumbents also figure their customers will gravitate toward a mix of private and public clouds. A retailer might keep its ordering system in-house and then rent extra computers during the holiday season to handle the shopping rush. The solution to that is called, of course, a hybrid cloud: a secure, reliable bridge between the data centers of IBM, et al., and those of their customers'.

The cloudpeople, particularly at Amazon and Google, tend to chuckle at the "private" and "hybrid" distinctions as the same play the big guys turn to every five years or so: Gin up a pitch that sounds good to cautious CIOs and will ensure another round of technology purchases.

"I think those companies are trying to hold onto what I believe will be a shrinking share of the market for business technology," says Google CIO Ben Fried. "When you hear companies talk about the private cloud, you need to understand: That is not the cloud."

The incumbents have a point, though: For larger corporations, going full cloud is not simple. These companies in all likelihood have spent decades buying mainframes, then minicomputers, then servers. Much work done on the older systems will probably remain there for years to come. Some may never make their way to the cloud. A company such as investment bank Credit Suisse (CS) has about 7,000 software applications developed over the course of 20 years. "It is unlikely those will go out to the cloud because the cost of migration is so high," says Steve Hilton, Credit Suisse's head of technology infrastructure services. He's skeptical about the cost savings of the Amazon-Google- Microsoft approach, particularly when it comes to mainstay business software. "I truly don't believe it's that dramatic," he says. "It's just a different consumption model, and I don't think it changes the price point by orders of magnitude."

Another argument the incumbents use concerns liability. Some companies justifiably fear they will be subject to lawsuits if they're not directly in charge of managing their own information. Imagine a health-care company trying to explain how it lost control of someone's records. The cloudpeople aren't so worried about that. Microsoft General Counsel Brad Smith says there are no "insurmountable" barriers to prevent companies or governments from moving sensitive data into the cloud. "The question is whether it could be made simpler and clearer and easier with new laws or regulations," he says. "I think the answer to that is almost certainly, 'Yes.'" Smith hopes lawmakers in the U.S. and Europe will have delivered firmer liability and privacy rules around the cloud by the end of 2012.

The familiarity and stability of traditional tech companies does seem to resonate with big customers. IBM, without disclosing exact numbers, says its cloud revenue doubled last year and should double again this year. It recently raised its 2015 cloud revenue forecast from $3 billion to $7 billion. AT&T has invested in building a more open system similar to Amazon's, and Verizon just paid $1.4 billion for Terremark Worldwide (TMRK), an infrastructure services company that made an early hybrid cloud push.

Scott Raney, a partner at venture capital firm Redpoint Ventures, which has invested in numerous cloud-powered startups, views the recent acquisitions and outpouring of rhetoric as a signal that the big boys fully appreciate what's at stake. Still, he can foresee their numbers dwindling as great volumes of data are sucked up into the cloud. The disaster scenario for the traditional heavyweights is that Amazon, Google, and Microsoft end up as the corporate information kingpins.

"There is one school of thought that the world is heading toward three really big data centers owned by those three companies," says Raney. "They will be the world's computers, more or less, and all the software will be running there. It's a pretty extreme view, but that's spooking the hell out of all the other companies."

There's a saying in the technology industry: People tend to overestimate what will happen two years from now and underestimate what will happen in 10. Well, the next 10 years have arrived. Silicon Valley veterans have long figured that once technology gets good enough, the world's computing infrastructure will come to resemble the electricity infrastructure—which is exactly what's happening. In his 2008 book The Big Switch, author Nicholas Carr documented how companies that once ran their own power generators eventually bought electricity from a select group of large providers because it was easier and cheaper to do so. The large providers then built enormous generators, aggregated demand, and invested more in fine-tuning their systems. Companies had to adjust to relinquishing control of part of their infrastructure, but once they did they were able to focus on getting better at whatever it was they made. Everybody won.

In the digital version of this shift, the role of utilities is played by the megadata centers. The cloudpeople are saying this technological wave, too, will make everyone a winner. Corporate technology has a chance to shift from a painful, dark art to something that injects new life into businesses. The only losers will be those companies that sit still and suffer grim outcomes at the hands of smaller companies that embrace the cloud.

Carl Ryden has issued this exact message to contemplative types at big companies. He is co-founder of MarginPro, an eight-person outfit in Charlotte that built a service out of Microsoft's Azure to help banks price commercial loans. Each month, MarginPro analyzes $700 million worth of lending. For every $10 million in revenue the company makes off this work, it pays $1,500 in cloud fees to Microsoft. "We sell around the technology guys and straight to the business folks," Ryden says.

MarginPro also analyzes its customer data to get a sense of trends in the loan market and the overall health of the economy. Ryden says he's impressed with the quality of the analysis and might turn it into an additional business. He's also delighted he now has the technological wherewithal that used to be available only to organizations with a lot more money—organizations such as banks, his clientele. "You can find 100 reasons not to move to the cloud," Ryden says. "But you're going to look up one day and all you will be doing is managing the systems that connect all your printers."