Google's secret of success? Dealing with failure

[DevilDog#1]

By Martin LaMonica, CNET News.com
Published on ZDNet News: March 3, 2005, 4:00 AM PT

BURLINGAME, Calif.--The technical wizardry behind Google's successful search engine may come down to a blindingly obvious insight: PCs crash.

On Wednesday, Urs Hoelzle, a vice president of engineering and of operations at the search giant, shed some light on how Google's data centers operate. Many people consider the company's operations expertise more valuable than the actual search algorithms that launched the enterprise.

Hoelzle spoke at EclipseCon, a conference for application programmers that's going on till Thursday here.

The way Google has been able to build out its computing infrastructure for millions, rather than tens of millions, of dollars is by buying relatively cheap machines. Looking at hardware costs, company engineers saw that purchasing a few high-end servers, with eight or more powerful processors, costs significantly more than dozens of simpler "commodity" servers.

The trick is to make these racks of hardware operate in tandem and to ensure that the failure of one machine does not derail an operation, such as returning a search query or serving up an ad.

Consider a home PC, Hoelzle said. Optimistically, a consumer PC might crash once in three years from a software glitch or hardware problem.

"At Google scale...if you have thousands of PCs, you can expect one (failure) a day," he said. "So you better deal with that in an automated way, or you will have service outages."

Google, known for its rigorous hiring practices aimed at attracting the brightest minds in computer science, has created a number of software tools to handle its computing installation.

The company wrote its own file system, called Google File System, which is optimized for handling large, 64 megabyte blocks of data. Significantly, the file system was designed to assume that a failure, such as a failed disk or unplugged network cable, can happen at any time.

Data is replicated in three places, and there is a "master" machine that


Click >here< to read more.

[DevilDog#1]

By Mary Jo Foley, Microsoft Watch
March 3, 2005

One of Microsoft's key Windows architects has defected to Google. But at least so far, no one is talking about what Marc Lucovsky's new role will be at one of Microsoft's major rivals.

A 16-year Microsoft veteran, Lucovsky was one of a handful of "Distinguished Engineers" at Microsoft. He is credited as one of the core dozen engineers that came from Digital Equipment Corp. to Microsoft and built the Windows NT operating system. He was charged with building the Windows NT executive, kernel, Win32 run-time and other key elements of the operating system. NT was the precursor to Windows Server.

In 2000, Lucovsky was named the chief software architect for Microsoft's .Net My Services (code-named "Hailstorm") effort. .Net My Services never materialized in the form that Microsoft originally envisioned—a set of personal Web services, hosted by Microsoft. Instead, the company has folded a number of the .Net My Services technologies into other Microsoft products.

Scripting.com's Dave Winer mentioned on his blog earlier this week that Lucovsky had defected to Google, with no further details.

Lucovsky "voluntarily left the company on 11/18/04," confirmed a Microsoft spokesperson. "Obviously Microsoft can't comment on whether or not he now works for Google."

Google officials did not return calls requesting comments on Lucovsky.