CNet has a great article on how Google works. Not the search algorithms but the actual hardware and software they run. Google keeps costs down by buying relatively cheap machines and writing their own software. “The company wrote its own file system, called Google File System, which is optimized for handling large, 64 megabyte blocks of data.”
They also plan for failures. They know that at least one machine a day will go down and their system was designed to take it in stride. Their data is stored in three different places to limit any type of failures and they have a “master” machine that can locate any of the copies when needed.
It also notes that Google’s machines run on a stripped-down version of Linux and have created a system to handle the massive amounts of data while returning your search query in a matter of seconds.
No matter how smart their system is though, they can’t prevent everything. “For all its built-in redundancy in case of failure, the system doesn’t address all problems, Hoelzle revealed. During the presentation, he showed a photo of six fire trucks responding to an emergency at a Google data center in an undisclosed location.”
This is a good article and you can read more about it in CNet’s story Google’s secret of success? Dealing with failure.
