Profile cover photo
Profile photo
James Pearn

Post has attachment
A vision of the future

I came across this image on the new Singularity Institute website and I really like it. Will the Earth ever look like this, I wonder? Imagine all of humanity condensed into a few mega high-rise cities while the rest of the planet is returned to nature.

I would expect the skyscrapers to be much taller though. The cities would be connected via a global network of 8,000 km/h vacuum tube trains. Food production would fully automated and energy would be generated via nuclear fusion.

Image created by: +Conrad Allan 

Post has attachment
The Transcension Hypothesis
When post-humans leave the visible universe

We've not yet managed to reverse engineer the brain and build artificially intelligent replicants, but it's only a matter of time. And once we do, an intelligence explosion will follow. Imagine building an artificial brain that is just as sentient as a human, but operates at a million times the speed and memory capacity?

These replicants will be our post-human descendants. With superintelligence they'll figure out a deeper understanding of the laws of physics. They'll then build technology to manipulate spacetime and disappear into whatever dimensions lie beyond. That's transcension.

Great new video from Jason Silva:
The Transcension Hypothesis - What comes after the singularity?

Post has attachment
What is Google's total computational capacity?

I estimate: 40 petaflops

This is 4x as powerful as the world's most powerful supercomputer.

For competitive reasons Google doesn't reveal this information themselves. We can, however, estimate their total number of servers together with the capacity per server. These figures can then be compared to other high-performance computer systems and used to extrapolate total capacity.

Number of servers

In a previous post from January 2012 ( I calculated that Google's total number of servers is around 1,800,000. This includes all eight of their self-built data centers currently in operation worldwide. Other respected industry watchers are saying Google has 900,000 servers ( But that figure is based on only a single data point (energy usage) that is both unreliable and over a year old. Google have opened whole new data centers since then. So I still think 1,800,000 is a reasonable up-to-date estimate.

Average power per server

In 2009 it was revealed that the average Google server is a commodity-class, dual-processor, dual-core, x86 PC system. That is, each server has four processor cores. See the paper where this is described: (PDF, page 7). Note that this paper was published three years ago. It's quite possible that the servers are replaced over a three-year cycle. So the average now, in 2012, might be a dual-processor, quad-core system (eight cores per server, or even more). But let's be conservative and assume the 2009 info is still valid.

This means Google is running ~7,200,000 processor cores.

Google has said they go for power in numbers. That is, they use lots of cheap processors rather than a smaller number of costlier, more powerful ones. Let's assume then that the average processor is one that first came to market five years ago, i.e. in 2007. This might be the Intel Core2 Duo E4400 ( running at 2 GHz. This processor is capable of around 6 gigaflops per core. Multiply that by our estimated number of cores and Google's total comes out at 43 petaflops.

The capacity of a system is not, however, a simple multiplication of core count and flops-per-core. Rarely can a system reach its theoretical maximum. So for that reason it's helpful to look at other large-scale systems where the total capacity is known.

TOP500 supercomputers

According to the list, the world's most powerful supercomputer is currently the K computer in Japan. It has 705,024 processor cores and a maximum speed of 10 petaflops. This gives it an average speed-per-core of 14.9 gigaflops.

The K computer uses Sparc VIIIfx processors which are rated at 16 gigaflops per core. This tells us that the supercomputer is achieving 93% of the theoretical capacity of all its processors combined. If Google's servers achieve a similar percentage that would mean their total capacity is 40 petaflops, or four times that of the K computer.

Note that even if Google were able and inclined to run the Linpack benchmark across their whole platform they still wouldn't qualify for inclusion in the TOP500 list. Supercomputers only qualify if they're housed entirely under a single roof.

Amazon EC2 Cluster

An Amazon EC2 Cluster instance is currently number 42 on the TOP500 list. Like Google, it is also built using commodity hardware. The exact details are not known, but their web pages mention Xeon and Opteron x86 processors. In a benchmark test the cluster was able to achieve 240 teraflops using 17,024 cores. This averages to 14 gigaflops per core. If Google's servers are around the same performance, that would give them a total of just over 50 petaflops.

Grid computing

BOINC is a grid computing system originally developed for the SETI@home project. Volunteers around the world download client software which utilizes their PC's spare CPU cycles for scientific research. As of February 2012 the system has ~450,000 active computers (hosts) and processes on average 5.7 petaflops.

If we assume that the average BOINC host has the same power as the average Google server, and if we also assume that the average BOINC host is utilized the same amount of time as a Google server, then we can simply multiply the figures. Google has four times the number of servers as BOINC has hosts, so that would mean Google's total processing power is 22.8 petaflops.

Folding@home is another distributed computing project similar to BOINC. It is designed to perform simulations of protein folding and other molecular dynamics. As of February 2012 the project had around 414,000 active processors for a total of 8.4 petaflops. If we assume that Google's average processor performs similar to the average Folding@home processor, this would bring Google's total processing power to 36 petaflops.

Future growth

If Google's computational capacity grows according to Moore's Law then it will double every 18 months. This means Google will become an exascale machine (capable of 1 exaflops) by 2019.

Google said themselves in 2009 that their system is designed for 1 to 10 million servers ( If they have ~2 million currently, that means there's room for five-fold growth, which would mean up to ~200 petaflops.

To reach 1 exaflops Google might need to evolve their architecture. Maybe that they'll start using GPUs, or processors with hundreds of cores. I've no idea, but I would guess someone inside Google is already thinking about it.


- - FLOPS on Wikipedia
- - K computer on Wikipedia
- - Amazon EC2 Cluster
- - Amazon EC2 Cluster
- - BOINC grid computing project
- - Folding@home grid computing project
- - the "globe" graphic used below

#googlecomputecapacity #googleservercount #petascale #exascale #exacycle #singularity

Post has attachment
How many servers does Google have?

My estimate: 1,791,040 as of January 2012
And projection: 2,376,640 in early 2013

This estimate was made by adding up the total available floor space at all of Google's data centers, combined with knowledge on how the data centers are constructed. I've also checked the numbers against Google's known energy consumption, and various other snippets of detail revealed by Google themselves.

Google doesn't publicly say how many servers they have. They keep the figure secret for competitive reasons. If Microsoft over-estimates and invests in more servers then they'll waste money - and this would be good for Google. Conversely, if Microsoft builds fewer servers then they won't match Google's processing power, and again, this would be good for Google. Nevertheless, from the limited amount of information that is available I've attempted to make a rough estimate.

First of all, here's some background on how Google's data centers are built and organised. Understanding this is crucial to making a good estimate.

Number and location of data centers

Google build and operate their own data centers. This wasn't always the case. In the early years they rented colocation space at third-party centers. Since the mid-2000s, however, they have been building their own. Google currently (as of January 2012) has eight operational data centers. There are six in the US and two in Europe. Two more are being built in Asia and one more in Europe. A twelfth is planned in Taiwan but construction hasn't yet received the go-ahead.

Initially the data center locations were kept secret. Google even purchased the land under a false company name. That approach didn't quite work however. Information always leaked out via the local communities. So now Google openly publishes the info.

Here are all 12 of Google's self-built data centers, listed by year they became operational:

2003 - Douglas County, Georgia, USA (container center 2005)
2006 - The Dalles, Oregon, USA
2008 - Lenoir, North Carolina, USA
2008 - Moncks Corner, South Carolina, USA
2008 - St. Ghislain, Belgium
2009 - Council Bluffs, Iowa, USA
2010 - Hamina, Finland
2011 - Mayes County, Oklahoma, USA

2012 - Profile Park, Dublin, Ireland (operational late 2012)
2013 - Jurong West, Singapore (operational early 2013)
2013 - Kowloon, Hong Kong (operational early 2013)
201? - Changhua Coastal Industrial Park, Taiwan (unconfirmed)

These are so-called “mega data centers” that contain hundreds of thousands of servers. It's possible that Google continues to rent smaller pockets of third-party colocation space, or has servers hidden away at Google offices around the world. There's online evidence, for example, that Google was still seeking colocation space as recently as 2008. Three of the mega data centers came online later that year, however, and that should have brought the total capacity up to requirements. It's reasonable to assume that Google now maintains all its servers exclusively at its own purpose-built centers - for reasons of security and operational efficiency.

Physical construction of data centers

Although the locations are public knowledge, the data center insides are still fairly secret. The public are not allowed in, there are no tours, and even Google employees have restricted access. Google have, however, revealed the general design principles.

The centers are based around mobile shipping containers. They use standard 40' intermodal containers which are ~12m long and ~2.5m wide. Each container holds 1,160 servers. The containers are lined up in rows inside a warehouse, and are stacked two high.

See the video Google released in 2009:

Are all of Google's data centers now based on this container design? We don't know for sure, but assume that they are. It would be sensible to have a standardised system.

As for the servers themselves - they use cheap, low-performance, open-case machines. The machines only contain the minimal hardware required to do their job, namely: CPU, DRAM, disk, network adapter, and on-board battery-powered UPS. Exact up-to-date specifications are not known, but in 2009 an average server was thought to be a dual-core dual-processor (i.e. 4 cores) with 16 GB RAM and 2 TB disk.

The containers are rigged to an external power supply and cooling system. Much of the space inside a warehouse is taken up with the cooling pipes and pumps. The cooling towers are generally external structures adjacent to the warehouse.

Counting servers based on data center floor space

This is by no means a precise method, but it gives us an indication. It works as follows.

First we determine the surface area occupied by each of Google's data center buildings. Sometimes this information is published. For example the data center at The Dalles is reported to be 66,000 m². The problem with this figure, however, is we don't know if it includes only the warehouse building itself or the whole plot of land including supporting buildings, car parks, and flower beds.

So, to be sure of getting the exact size of only the buildings, I took satellite images from Google Maps and used those to make measurements. Due to out-of-date imagery some of the data centers are not shown on Google Maps, but those that are missing can be found on Bing Maps instead.

Having retrieved the satellite imagery of the buildings I then superimposed rows of shipping containers drawn to scale. Care was taken to ensure the containers occupied approximately the same proportion of total warehouse surface area as seen in the video linked above. That is, well under 50% of the floor space, probably closer to 20%. An example of this superimposed imagery is attached to this post, it shows one of the warehouses in Douglas County, Georgia, USA.

Having counted how many container footprints fit inside each warehouse, I then doubled those figures. This is because I assume all containers are stacked two high. Quite a large assumption, but hopefully a fair one.

It turns out that in general the centers house around 200,000 servers each. Douglas County is much larger at about twice that figure. Meanwhile Lenoir, Hamina, and Mayes County are smaller. Mayes County is due to be doubled in size during 2012. The sizes of the future data centers in Singapore and Hong Kong have not been measured. Instead I assume that they'll also host around 200,000 servers each.

This results in the following totals:

417,600 servers - Douglas County, Georgia, USA
204,160 servers - The Dalles, Oregon, USA
241,280 servers - Council Bluffs, Iowa, USA
139,200 servers - Lenoir, North Carolina, USA
250,560 servers - Moncks Corner, South Carolina, USA
296,960 servers - St. Ghislain, Belgium
116,000 servers - Hamina, Finland
125,280 servers - Mayes County, Oklahoma, USA

Sub-total: 1,791,040

Future data centers that'll be operational by early 2013:

46,400 servers - Profile Park, Dublin, Ireland
200,000 servers - Jurong West, Singapore (projected estimate)
200,000 servers - Kowloon, Hong Kong (projected estimate)
139,200 additional servers - Mayes County, Oklahoma, USA

Grand total: 2,376,640

Technical details revealed by Google

A slide show published in 2009 by Google Fellow +Jeff Dean reveals lots of interesting numbers. In particular it mentions "Spanner", which is the storage and computation system used to span all of Google's data centers. This system is designed to support 1 to 10 million globally distributed servers.

Given that this information was published over two years ago, it's likely the number of servers is already well into that 1-to-10 million range. And this would match with the floor space estimation.

Counting servers based on energy consumption

Last year +Jonathan Koomey published a study of data center electricity use from 2005 to 2010. He calculated that the total worldwide use in 2010 was 198.8 billion kWh. In May of 2011 he was told by +David Jacobowitz (program manager on the Green Energy team at Google) that Google's total data center electricity use was less than 1% of that worldwide figure.

From those numbers, Koomey calculated that Google was operating ~900,000 servers in 2010. He does say, however, that this is only "educated guesswork". He factored in an estimate that Google's servers are 30% more energy efficient than conventional ones. It‘s possible that this is an underestimate - Google does pride itself on energy efficiency.

If we take Koomey's 2010 figure of 900,000 servers, and then add the Hamina center (opened late 2010) and the Mayes County center (opened 2011) that brings us to over a million servers. The number would be ~1,200,000 if we were to assume all data centers are the same size.

Koomey's study:


The figure of 1,791,040 servers is an estimate. It's probably wrong. But hopefully not too wrong. I'm pretty confident it's correct within an order of magnitude. I can't imagine Google has fewer than 180,000 servers or more than 18 million. This gives an idea of the scale of the Google platform.


YouTube videos:
- - Google patent for container-based data centers - Standard container sizes - +Jeff Dean's slideshow about Google platform design - “In the Plex” book by +Steven Levy - +Jonathan Koomey's data center electricity use

Articles by +Rich Miller of Data Center Knowledge:

Original copy of this post:

Attached image below is one of Google's data warehouses in Douglas County, Georgia. Photo is from Google Maps, with an overlay showing the server container locations.
Commenting is disabled for this post.

Post has attachment
Norwegian airlines now have free onboard Wi-Fi. I took this selfie moments ago at 11,000 meters en route from Stockholm to Munich.

Post has attachment
If you're travelling through Munich Airport in the next 10 days it is worth scheduling an extra hour to watch the artificial surf wave. Details:

Post has attachment
Currently on tour in Scotland. Frustrated to have limited internet just as Google unveils the greatest thing since sliced bread. :-)
Wait while more posts are being loaded