It may be the start of a new trend, software that automatically upgrades itself silently in the background without ever bothering users. Google has been doing it successfully with its Chrome web browser, and soon Mozilla will jump on the bandwagon with Firefox.

You may love it or hate it, but for most users, software that automatically upgrades itself can be a blessing, and in more ways than is immediately apparent.

And it’s not just great for users, it’s great for developers because it allows them to innovate and develop at a fast pace, pushing out frequent updates without annoying their users with upgrade notices. In short, automatic upgrades let developers push the pace of innovation.

Case study, web browsers

The perfect case study for automatic upgrades is Google Chrome, which has been developed at a break-neck pace for the last couple of years. The frequent updates to the Chrome browser would have driven its users mad if they had to keep up manually.

As you can see in the above graph (aside from the rapid growth in market share), each new version replaces the previous one very rapidly. So the situation for Google Chrome now is that all its users are running version 5. If Google hadn’t adopted automatic upgrades, there would be users running all five different versions, not to mention subversions of each. It would be a mess.

Now take the other end of the spectrum, where Microsoft hasn’t been able to convince a massive user base to get rid of older versions of Internet Explorer. They have all that legacy software to deal with, and web developers have to deal with that as well.

Which leads us to an important point:

Don’t forget the secondary effects of having massive amounts of legacy software in circulation: In the specific case of web browsers, a lot of innovation on the web currently depends on browser features and new developments in that field (HTML 5, CSS 3, etc). Having a ton of legacy browser software out there is effectively hampering innovation since developers cannot assume that the features they need will be available.

Pros and cons of automatic upgrades

We’re assuming that the upgrade process is silent and takes place in the background without asking the user for approval except when the software is first installed.

Pros

• Silent upgrades mean users don’t have to be annoyed with frequent upgrade messages. Great for users, and great for developers since they can develop the product at a rapid pace. This should have a positive effect on innovation.
• Users will always be running the latest, best version.
• Developers don’t have to support old versions of the software.
• Since developers can assume that users will be running the latest version, they can USE the latest features and assume they are there, which helps them innovate.
• Old software can be a security threat if left unpatched.

Cons

• Users have to give up some control.
• Some potential for abuse from malicious developers.

That’s about it. Less control is the only real con for users. The abuse potential is there, so we had to mention it, but if you think about it, it’s there with manual upgrades as well. The moment you install software on your computer, you are inherently trusting that it won’t do something nasty. Why not take the full step and let the software keep itself up to date as well? Out of sight, out of mind, and let you get on with your life.

Like web apps

When you think about it, automatic upgrades mimic the way web apps work, where a new version takes effect immediately. Users are always running the latest and greatest version of the software.

And if we may be so bold as to make a prediction, this is where we are heading with almost all types of software. It’s a paradigm shift in the making. There may be some resistance for a while from users who like to have full control, but eventually the sheer convenience of this model both for users and developers will be too much to ignore. It’s simply too practical. We suspect that silent, automatic software upgrades will be a full-grown trend within two or three years.

And guess what? When that happens, the pace of innovation will increase. It’s almost inevitable. Old software will be phased out faster, no longer dragging down progress.

Yes, newer isn’t always better, but it almost always is. Compare the software of today with that from 10 years ago. Which would you rather be using?

The two mobile platforms with the most apps are Google’s Android with around 95,000 apps, and Apple’s iOS with around 250,000 apps.

Those are impressive numbers, but this article isn’t about the sheer number of apps available. Instead, we wanted to focus on a very interesting distinction between the two platforms: The radical difference in the ratio between free and paid apps.

Free versus paid

They say that a picture is worth a thousand words, so let’s illustrate the difference between the two app stores with a picture and save us some writing:

That’s quite a difference, isn’t it? On Apple’s App Store, roughly 70% of the apps are paid apps. On Google’s Android Market, it’s almost exactly the other way around, 64% free apps.

Why so many more free apps on Android?

Or from the other point of view, why so few paid apps on Android compared to iOS?

It’s certainly not for a lack of developers. There are surveys indicating that there are now more Android developers than iPhone developers.

But the fact remains: Far fewer Android developers are trying to sell apps compared to their iPhone counterparts.

There may be several reasons for this:

• Money. Many developers are uncertain if Android is a lucrative enough market (we wrote about one aspect of this last week), although there have been voices raised to the contrary recently. The iPhone (and iOS), on the other hand, is an established platform with a number of high-profile success stories and may therefore to a greater degree be attracting developers wanting to make a profit.
• Many developers are shut out from selling apps. Only developers from nine countries are allowed to sell apps on Android Market. Developers in other countries may therefore be much more likely to release applications for free instead of trying to earn some money from selling at a modest price.
• More hobbyists? Perhaps the lack of an approval process makes it more likely for hobbyist projects to exist on Android.
• Different culture? It’s quite possible that Android has attracted a higher share of developers from the open source and Linux world, who are used to making their applications available for free.

What do you think? Is there any one big reason why the ratio between free and paid apps is so very different on Android and iOS?

Data sources: App Store numbers from 148Apps.biz, Android Market numbers from AndroLib.

Do you run a web service or hosting company? Do you like transparency? Then this might be of interest to you.

Service status blogs are becoming increasingly common these days and are usually very appreciated by users. Look no further than Twitter’s famous status blog, or the Google Apps status page. Status blogs (or “status pages”, depending on approach) may look and work differently, but they all serve the same purpose, informing users about service issues.

Now it’s easier than ever before if you want one, or want to make your existing status blog even better.

Making it easy to create a powerful status blog

To make it really easy for companies to offer a comprehensive status blog, we are now releasing a free Pingdom status plugin for Wordpress. Using this plugin you can combine Pingdom’s monitoring with the world’s most widespread blogging platform.

What the Pingdom status plugin adds to your blog:

• A public status page. It creates a page on your Wordpress blog which shows the current status (up or down) of the sites and servers you monitor with Pingdom.
• Monitoring history. Users can drill down and view detailed uptime and response time history for each monitored item.
• Make all or just some results public. You can select to make all your monitoring results public, or just some of them. Often companies have additional monitoring set up that may not be directly relevant to their users.
• Grouping. You can create groups of monitored items and name them, for example “web servers,” “database servers,” “websites,” etc. This makes it easy for you to categorize the results for your users.
• Default or custom design. The plugin comes with an attractive default styling, but you can modify it as much as you like, for example by adding your own colors and branding, etc.

All you need is a Pingdom account and a Wordpress blog. Once you have that, setting up the plugin won’t take more than five minutes.

To get started, just follow the plugin installation instructions. All settings are done inside the standard Wordpress admin control panel, in the Plugins section.

Screenshots

Here is an example of what the main status page looks like:

In the above case we have created a group named “Websites.” You can of course have multiple groups.

By clicking on any of the monitored items, users can drill in and examine its status history:

Final notes

This plugin was designed for self-hosted Wordpress blogs (i.e. using the Wordpress.org software). It won’t work for blogs hosted on the Wordpress.com service.

Also, a small but important thing to keep in mind: Host your status blog separately. If your main site or service goes down, you still want your status blog to be accessible.

And in case you missed the link to the plugin, here it is again: http://wordpress.org/extend/plugins/pingdom-status/

We hope you will find it useful.

About Pingdom:

In case you arrived here not familiar with Pingdom, we offer a service that monitors websites and servers for our users, sending alerts if errors are detected, and providing a control panel where our users can examine their monitoring history and troubleshoot problems. We have tens of thousands of users, ranging all the way from large corporations to hobbyist webmasters. You can read more about our features at www.pingdom.com.

Pingdom has paid monitoring packages starting at $9.95 per month, but also a free package for monitoring one site.

From its official launch in October 2009, it took Windows 7 only nine months to pass Vista. Now the next question is when it will catch up with Windows XP. Because, unbelievable as it may seem, Windows XP still has a massive 55% of the desktop OS market. That is more than Windows 7 and Vista combined.

To figure out when Windows 7 will overtake XP, we have made a prediction based on the average market share changes over the past six months. It will give us an idea of what will happen if things continue at their current pace.

Predicting future Windows market share

Please note that everything past August 2010 in the below graph is an estimate. The data until today is actual global market share data from Statcounter. We have marked the dates when Windows 7 caught up with Windows Vista, and when it will catch up with Windows XP if the current trend continues.

Observations, based on the current rates of change:

• Windows 7 will overtake XP by the end of 2011.
• Windows Vista may die out sooner than Windows XP, in spite of being a much younger OS.

Other factors to consider

We based our prediction on past market share data, but that is of course a simplified view of something that depends on a number of factors, many of which change over time.

One major factor that may speed up the adoption of Windows 7 and the decline of Windows XP is Microsoft itself. The company is actively discouraging the use of XP and is trying to speed up its demise. We suspect these efforts will escalate the longer XP is able to hang on to a large share of the market.

Something else to take into account is that corporate users tend to wait at least a year before they migrate to a new OS, so we may see a speed-up in Windows 7 adoption late in 2010 or early 2011 as the corporations start to jump aboard. Since many of them will be migrating from XP, not Vista, this may speed up the decline of XP even further.

The current situation vs. the future

For the curious, here are the current desktop OS market shares for the different Windows versions:

• Windows XP, 55.6%
• Windows Vista, 17.8%
• Windows 7, 19.1%

We were right about when Windows 7 would overtake Vista, but that was a much easier thing to predict. If the above prediction holds true, however, Windows 7 will become the dominant version of Windows roughly two years after its launch. Perhaps it’s finally time for XP to retire?

Market share data source: StatCounter (only the historical data, not the prediction).

Although the growth of Firefox has stagnated a bit lately due to the increasing competition from rival browsers, it’s still one of the biggest success stories in the history of the Internet and has the second-largest user base of any web browser.

Firefox has a widespread global user base, but we wanted to find out where it is most common, or another way of looking at it: how are the Firefox users distributed?

Luckily for the curious, Mozilla has a page that shows continuously updated download stats for the latest major version of Firefox. This is currently Firefox 3.6, which went live on January 21 this year. All numbers in this article are therefore for Firefox 3.6.

We didn’t just examine where Firefox has the most downloads, we also took some time to calculate the relative popularity of Firefox in these countries. (Because we could.)

Summary

Before we head on to the charts, here is a brief summary of what we found out.

• Worldwide Firefox 3.6 downloads so far: 126.5 million
• Countries where Firefox 3.6 has been downloaded more than 1 million times: 26
• Top 5 countries in terms of downloads: United States, Germany, Poland, Russia and Brazil.
• Top 5 countries (out of the ones with 1M+ downloads) when you count downloads per Internet user, i.e. “popularity”: Finland, Poland, Romania, Germany and Australia.
• The United States accounts for 15.7% of all Firefox 3.6 downloads. Germany accounts for 9.2%.

The data was collected this Monday. We thought that was worth mentioning since it’s a moving target as downloads keep ticking.

Top countries in terms of Firefox downloads

It’s important to note that download numbers will not be equal to user numbers (those are not available). That said, the download numbers will give us a very good indication of where Firefox is used the most.

As you can see, the United States and Germany are way ahead of the other countries when it comes to downloads. One reason for this is that these countries have very large Internet populations. The United States has 239 million Internet users, and Germany has 65 million. As you will see here below, though, Firefox is much more popular in Germany than it is in the Unites States.

Popularity

If you take the number of Internet users per country into account, you can find out the relative popularity of Firefox in each country.

We essentially looked at downloads per Internet user, but to make the numbers a bit easier to digest, the chart shows how many Firefox 3.6 downloads there have been for every 1,000 Internet users each country has. The higher the number, the more popular Firefox is. We only included countries that have more than 1 million Firefox downloads, just like in the previous chart.

So, although the United States accounts for the highest number of Firefox downloads by far, when you take its size into consideration, the actual penetration is much lower than in many other countries. If Firefox were as popular in the United States as it is in Finland, US downloads would have more than tripled.

Final words

Since we looked at the latest version of Firefox, what you see in this article is an up-to-date view of which countries are the big Firefox users of the world right now. We hope you found this little study interesting.

And a small side note… It would have been very cool to be able to compare download numbers for all the different web browsers that exist out there, but unfortunately not everyone is quite as open as Mozilla about these things. Are you reading this, Google, Microsoft and Apple…?

Data source: Firefox download numbers from Mozilla.

Google has made great strides with Android, and a ton of developers have flocked to the growing mobile platform. Not everything is rosy, though. One major concern among developers is that piracy levels are very high on the platform.

Google is of course not oblivious to this and recently announced plans to combat piracy with DRM methods that app developers can include in their apps. But there is one problem that is arguably much more problematic for Android developers when it comes to getting paid for their apps, and it isn’t getting nearly as much attention as we think it should.

The big problem with selling Android apps

Google is talking about fighting piracy, but perhaps the first thing they should focus on is actually making it possible for users to buy apps. All users. Sounds rather logical, doesn’t it? So what are we talking about? The problem lies with Android Market.

You can only pay for apps in 13 out of the 46 or so countries where Android phones are available. For those of you who like stats, 13 in 46 works out to less than 30%. Contrast this with Apple’s App Store, which supports paid apps in 90 countries. This is a huge advantage iPhone developers currently have over Android developers.

This is, in our opinion, one of the main reasons why piracy is running rampant on the Android platform. If a large portion of the world’s Android users can’t even pay for apps, is it so strange that some of them turn to piracy?

In other words, piracy isn’t the root of the problem, the inability to pay is.

This is of course bad news for Android app developers because:

• It will result in fewer sold apps (obviously).
• It may also get customers used to not paying for apps.

Let’s expand upon that second point, because it’s important.

Fostering a culture of free

We all like free, right? But the reality is that it can be bad news for developers if that mentality goes too far.

If Google doesn’t quickly make it possible for users in more countries to easily pay for apps, the company may create a long-term problem. People in those countries will simply get used to pirating their apps. They will get used to all Android apps being “free.”

So what happens once these users finally have proper access to paid apps? Sure, some of them will be paying, if nothing else because it’s more convenient, but the risk is that a significant portion of users will not like the idea of suddenly paying for something that so far has been available at no cost. Google will effectively have created “pirates” out of people who may otherwise not have gone down that route.

To say that this would severely hinder Android developers from making a living is an understatement.

The situation today

Paid Android apps are available in the following countries: Australia, Austria, Canada, France, Germany, Italy, Japan, Netherlands, New Zealand, Spain, Switzerland, United Kingdom, and the United States.

This means that big parts of Europe (for example all Scandinavian countries) are left without paid apps, all of Asia with the exception of Japan, and Latin America. Those are some pretty large markets to leave out.

We’re based in Sweden, one of the countries currently left out. Android is becoming a pretty popular platform here, but there is also widespread frustration among users that paid apps are not readily available. These are people who actually want to pay for apps, but have to jump through hoops to do it or turn to piracy. NOT good for business.

Facebook recently passed an incredible milestone, 500 million active users. And it keeps growing.

Although Facebook initially focused on the United States, it soon turned its eyes towards the horizon and the rest of the world. And it’s a tactic that’s been working extremely well. Facebook’s current user base is more than 1.6 times the size of the entire US population. If Facebook were a country, it’d be second only to China and India.

And speaking of countries… Now that it’s gone global, which countries have the most users on Facebook?

Top countries on Facebook

Since we don’t have actual user numbers and demographics from Facebook itself (only Facebook has those), we looked at per-country traffic estimates to Facebook.com. The result is quite interesting. (We used data from Google Ad Planner, which provides traffic stats for a large number of sites.)

These were the top 10, but there are of course a lot of countries that have a ton of Facebook users. To continue the list, the countries ranking 11-20 are:

Malaysia (12 million), Spain (12 million), the Philippines (10 million), Australia (9.1 million), Argentina (8.2 million), Taiwan (8.2 million), Colombia (7.5 million), Brazil (6.2 million), Chile (6.2 million), Thailand (6.2 million).

Estimated monthly visitors to Facebook.com shown in parenthesis.

Additional observations

Google Ad Planner estimates worldwide monthly visitors to Facebook.com to 550 million. This is slightly higher than the 500 million active users Facebook says it has, but on the other hand, not all visitors to Facebook.com need to have an account. Still, we’ll use the 550-million number in the calculations here below to be consistent (i.e. to avoid mixing data from different sources, which can give misleading results).

• The United States alone accounts for almost 24% of Facebook’s users.
• The United States has 4.6 times as many Facebook users as the second-largest country on Facebook, the United Kingdom.
• The top 10 countries on Facebook account for almost 58% of its users.

The rest of the countries account for the “long tail” that makes Facebook the colossus it has become.

Final words

Facebook’s international expansion is well under way, and it’s arguably the largest and most wide-spread social network that ever existed. As Facebook grows on a global level, the current US dominance of the social network will gradually diminish and become more similar to the general distribution of Internet users.

Mark Zuckerberg has stated that it’s almost a guarantee that Facebook will one day hit 1 billion users, and at the current rate, it’s certainly looking possible. Considering there are close to 2 billion Internet users in the world, there’s still plenty of room to grow.

Linux has been around for almost two decades now. It has become a resounding success as a server OS (for example as the L in the famous LAMP stack), and more recently as a mobile OS (Android). But what about on the desktop?

Linux enthusiasts have been predicting the rise of Linux as a desktop OS for the better part of a decade. To name just one of many examples, in 2003, Siemens Business Systems predicted that Linux would have captured 20% of the desktop market by 2008.

Well, it’s now 2010, and desktop Linux isn’t even close to 20%. Or one tenth of that.

Linux desktop OS market share

The sad truth is, Linux on the desktop has so far failed to really take off. Linux hasn’t even been able to claim a single percent of the desktop OS market.

Here is the current desktop OS market share distribution, courtesy of StatCounter:

As of July 2010 Linux had less than one percent of the desktop OS market. Perhaps this is a temporary low?

Unfortunately not. Here below you can see the last two years of Linux’s desktop OS market share. Things haven’t been moving much.

We should point out that there will be slight variations in the estimated market share depending on the source and the method used for acquiring the data. Other sources than StatCounter will show slightly different market share data, but it’s always around one percent. The important point here is that the market share is tiny, and isn’t really going anywhere.

What about Ubuntu?

Ubuntu’s rise to fame since its launch in 2004 has been remarkable. It has become the largest Linux desktop OS distribution by far, but the question is, why hasn’t it been even more successful?

Even with the relative success of Ubuntu, it looks like it may have mostly cannibalized the market share of other Linux distributions instead of bringing new people to Linux.

What can be done to grow Linux on the desktop?

Here are some things that we think could be done to make Linux more popular on the desktop. This would demand the support of a more or less unified Linux community, so it might be unfeasible, but one can dream, right?

• Center around one Linux desktop distribution.
• Center around one GUI.
• Polish the product (yes, you have to think of it as a product) until it shines, and make sure there are plenty of polished apps to go with it.
• Be fanatical about consistency and usability.
• And this may be the hardest one of all: Don’t build for Linux’s current hardcore audience of tech geeks. If the goal is to bring in new users, that approach is a big no-no since it will alienate the vast majority of the population.

Then of course there’s the marketing aspect of it, branding, building mind share, and getting manufacturers to include Linux as a viable option on their computers.

Final words

Some of the charm of Linux is that it is so flexible, and that you have so many choices available to you, but this can also be a curse when it comes to mainstream adoption. Although Linux on the desktop is a far better experience these days than it used to be, it hasn’t been able to catch up with the other OSs out there that provide a more unified front to the end user.

Linux is not in any danger of going away, so it’s quite possible that all of this doesn’t matter. Plenty of users are huge fans of the OS. Several people here at Pingdom use and are quite happy with Linux (primarily Ubuntu). Perhaps Linux will just continue to be a small niche desktop OS. There’s nothing wrong with that, not everything has to go mainstream.

(And in the meantime, people will continue to predict the rise of Linux on the desktop.)

About the stats: The OS market share data from StatCounter is based on visitor stats from more than three million websites.

Downtime sucks. When your website is down, it might as well be invisible. That alone should be reason enough to monitor your website’s availability, but just to really push home this point, here are another 19 excellent reasons why you should monitor your website.

1. All websites run into trouble from time to time. Murphy’s Law will make sure of that. There simply are too many things that can break a website, from server hardware issues to software issues to coding errors to network downtime to power outages to… The possibilities are, as the saying goes, endless.
2. The sooner you know about a problem, the sooner you can fix it. Conversely, if you don’t know about a problem, you can’t fix it.
3. It’s embarrassing when your users call you to let you know your site is down. You shouldn’t let your users or customers act as your monitoring. If they know before you do, that can easily get very embarrassing.
4. Many web hosting companies won’t be upfront about site issues unless you have proof to back it up. (Other web hosts might thank you for bringing problems to their attention.)
5. Think about your reputation. Outages are often discussed in public, and with less-than-kind words.
6. Is your website becoming more or less stable over time? There is no way you can know that without monitoring.
7. Stop guessing and assuming. By monitoring your website you arm yourself with actual knowledge instead of having to rely on guesswork and estimates. Get the facts straight.
8. No alerts = calm waters and peace of mind. You will know if your website is working even if you’re on the road, at the beach, or simply away from your desk for a few hours. Besides, you don’t actually visit your website every minute of the day, do you?
9. Your website may work for YOU, but what about the rest of the world? Network peering issues can cause widespread problems on the Internet.
10. Ok, so your website was down. But WHY? A good monitoring service can help you find out.
11. There is no “in the middle of the night” on the web. Downtime at night in your time zone is downtime during office hours in some other place.
12. Your web host may have an uptime guarantee, but it is often just a network uptime guarantee. It doesn’t guarantee that your website or server will work, and those two you can screw up yourself by accident or by getting a spike in visitors.
13. It doesn’t have to cost anything. There are free monitoring options out there, including from us here at Pingdom (pardon the shameless plug, but have a look at Pingdom.com/free if you’re interested).
14. It’s easy. Setting up monitoring isn’t rocket science, and with today’s online monitoring services you won’t need to install anything.
15. It’s a tiny investment of time that will continue to pay off. You set it up once, and then it keeps running without any need for intervention from you, continuously testing your site for you. Look at it this way; if you can spend five minutes to do something that will alert you whenever you have site issues, all year round, that time is extremely well spent.
16. Think of it as a cheap but effective insurance against embarrassingly long downtimes. It helps you as a webmaster to troubleshoot, and it helps keep your web host honest.
17. You’ll have proof that you’re doing a good job. If you work with operations for a web service, you’ll have proof that you are doing well. Which leads us to…
18. You can brag about your reliability. Has your site been up for two months straight? Be proud. Show your users your uptime numbers.

Considering the business we’re in, we here at Pingdom may be biased, but we strongly believe that all webmasters out there should monitor their sites. We wouldn’t be in this business otherwise, and we do know what we’re talking about.

Most responsible webmasters do backups. Those same webmasters should be just as careful about monitoring, because it’s just as important. (That was reason number 20, by the way. )

No wonder many bloggers have a hard time getting noticed. According to Blogpulse, there are more than 144 million blogs in the world, publishing 1 million posts per day. So there is some competition.

That’s an awful lot of hay for potential readers to sift through for that one single needle that your blog represents.

Thankfully, it’s not quite as bad as it looks. Keep in mind that a significant portion of those 144 million blogs are:

• Spam blogs.
• Blogs that are rarely updated.
• Blogs outside your genre.

On the other hand, that will still leave you with anywhere from thousands to hundreds of thousands of blogs to compete with (depending on what you’re blogging about). Quite a challenge.

So, best of luck to all you bloggers out there, and keep typing.

Photo by Maurice Koop.

Google has been dominating the search engine market for years, but at least there are some competitors that have a few percent each.

But if you look at mobile search, i.e. search on mobile devices, which is more or less the smartphone market, Google is utterly crushing the competition to a level that it’s never managed in the regular search market.

Just look at this very telling chart, showing Google’s overall search and mobile search market shares in relation to those of Yahoo and Bing, its two closest rivals. (These are global stats.)

Above: The search market share does NOT include mobile search (confirmed with Statcounter).

Pretty incredible, isn’t it?

There are two things we’d like to emphasize here. They are two sides of the same coin, but worth pointing out separately:

1. Google’s mobile search market share dominance is almost complete. The company has made an incredible land grab in the mobile sector. Compare this to the other search engines. None of them have managed to claim even one percent of the mobile search market.
2. Google’s piece of the mobile search pie is even larger than their already impressive share of the overall search engine market. For Yahoo and Bing, the situation is the opposite. Their mobile efforts are nothing compared to their search engine market share.

If Google firmly believes that mobile is the future (which is the opinion of CEO Eric Schmidt), they are making all the right moves.

And Google’s mobile dominance has been increasing. One year ago, its share of the mobile search market was 95.58%. That’s significantly less than today’s 98.29%. Who knows, in a few months, perhaps they will pass 99%. At this point this actually seems plausible.

What can change?

Needless to say, Android is Google territory, so they have that market gobbled up. Google will remain the search engine of choice on that platform.

Then there’s the iPhone, which is another story entirely. Google’s grip on mobile search could be broken if Apple kicks Google out of the iOS platform. Even though the companies are currently rivals in the mobile market, we don’t think that is likely to happen. For one, we suspect users would be too annoyed for Apple’s liking.

Another thing that could potentially happen is that Windows Phone 7 becomes a huge success (something we’re skeptical to at this point in time), which would boost Bing’s market share since Microsoft would of course put its own search engine as the default in its own mobile OS.

Conclusion

Google seems to have taken the necessary steps to bring its search market dominance over to the mobile sector. This much thanks to its early collaboration with Apple and later secured by Android, which ironically now is the very thing that threatens Google’s dominance since it has made Apple a rival.

But for now, Google reigns supreme.

A side note on the data: This data comes from StatCounter, and is global data based on information from visitor stats from more than three million websites.

The world is a big place, but so is the Internet. We know which countries are the largest in the real world, but what about on the Internet?

This article examines which countries are the largest in terms of Internet users, and also looks into their growth potential. That last point is very interesting to look at because it’s an indication of how the power balance on the Internet might shift in the future.

But before we head on to the charts, let’s start with a few interesting findings.

Observations on Internet usage

Here are some standout facts and observations that give additional perspective to the Internet usage of the top countries on the Internet.

• There are a total of 1.8 billion Internet users in the world.
• There are 32 countries with more than 10 million Internet users.
• The top 10 countries on the Internet together have 1.17 billion Internet users. That’s 65% of all Internet users in the world.
• The top 20 countries on the Internet together have 1.47 billion Internet users. That’s just under 82% of all Internet users.
• India is the fourth largest country in terms of Internet users in spite of having an Internet penetration of a measly 6.9%. This thanks to its huge population.
• China takes the top spot both in terms of population and Internet users. China has almost twice (1.8x) as many Internet users as the United States.
• China together with the United States, the top two countries, make up half of the Internet users in the top 15.
• Out of the top 20 countries, the five with the highest Internet penetration (not users) are: United Kingdom (82.5%), South Korea (81.1%), Germany (79.1%), Japan (78.2%), United States (76.3%).

By Internet penetration, we mean the share of the population made up of Internet users.

Top countries in terms of Internet users

The Internet is almost like its own world, so when you’re thinking about what countries are the largest in that world, it makes sense to count Internet users instead of real-world population.

Not only is China the largest in terms of real-world population, the country also has by far the largest number of Internet users. This may not come as a huge surprise, but looking past the absolute top of the list, there are some countries that may be unexpected residents in the top 20.

A few additional observations:

• The tenth largest country on the Internet is Nigeria. So although Africa as a whole is often considered to be a bit behind the curve on the Internet, it does have a presence among the top countries.
• Seven out of the top 20 countries are Asian (35%).
• Five out of the top 20 countries are European (25%). Six (30%) if you also count Russia.
• Three of the top 20 are English-language countries (four if you count India).

Internet users versus country populations

We thought it would be interesting to show you the number of Internet users these countries have in relation to their actual population size. The below chart clearly shows how much the Internet penetration differs between countries.

Why even look at this? One word: potential.

This chart identifies which of these countries have a lot of growth potential on the Internet, and which don’t.

Note: This chart is still sorted by Internet users, so it has the same order as the previous chart.

Countries where Internet users make up a minority of the population have a larger growth potential than countries where Internet users are in majority. This has big implications for the future.

Many of the more developed nations already have a high Internet penetration, having already converted most of their population to Internet users. Look at the United States, Japan, Germany, the United Kingdom, and so on. Their size on the Internet has little room for growth.

But look at countries like India, China, Brazil, Vietnam, the Philippines, Russia. As the Internet penetration in these countries rise, the balance of power on the Internet will start to shift.

In five to ten years, the order in the top 20 list will be quite different.

Final words

The Internet is a truly global phenomenon (which probably hasn’t escaped anyone), so it’s nice to see countries from all parts of the world in the top 20 we presented here.

We hope we have given you some insight into what the Internet of today looks like in terms of Internet users and nationalities, and hopefully also some food for thought about the future.

Data source for Internet and population numbers: Internet World Stats

Not to beat a dead horse, but why didn’t Apple think of this…?

DNS, the Domain Name System, is a vital part of the Internet. And since it’s such a vital part of the Internet, it’s important that websites have correct DNS settings. If they don’t, it can lead to a number of problems, one of them being downtime. On top of that, bad DNS settings can be hard to track down and can cause a lot of head ache for webmasters and site owners.

We here at Pingdom run a website monitoring service that tracks the uptime of tens of thousands of websites for our users, so we deal with site issues on a daily basis. Over time, it has become exceedingly clear that a large portion of the various errors we detect are caused by bad DNS settings or poorly configured DNS servers.

This is why we now are introducing an addition to our free webmaster tools: a DNS health test.

Testing your website’s DNS settings for problems

This new test takes a domain name – that of your website, presumably, but you can test any domain name – and figures out which DNS servers it’s set up on, then makes sure that there are no strange settings that can cause problems.

It’s a very thorough test, so it should root out any issues that can cause you problems either now or down the line.

Above: You want it to look like the left one, not the right one…

Those of you who like to know exactly what tests are being done can find the details under the “Advanced View” tab.

Share the results, get help

If you get warnings or errors from the test, make sure to bring this to the attention of the person or company that administrates your DNS servers and settings (if it’s not you).

For this purpose, we automatically create a permalink to your test that you can copy and paste into an email or other message. It shows up as soon as the test has finished. If you’re feeling social, you can also share the results directly to Twitter or Facebook.

In some cases, warnings in this test will be things you can live with, but it’s best to be sure. Ideally, your test should be all green and that is what you should strive for.

Final words

The DNS Health test is our latest addition to Pingdom Tools, which also includes Ping and Traceroute tools, and the popular Full Page Test that helps you track down bottlenecks with your website’s performance.

Pingdom Tools is a long-term project for us, and we want it to be a useful and free resource to you, the webmasters, sysadmins and site owners of the Internet. We hope you will find it useful.

We would also like to give a big shout out to the DNS gurus over at .SE, whose work this latest addition is based on.

And now, what are you waiting for? Go test your DNS settings!

Google is a bit of a paradox. On one hand, it promotes openness, with a stated mission to “organize the world’s information and make it universally accessible and useful.” An admirable goal. On the other hand Google is often secretive when it comes to publicly sharing information about its own websites and services.

Nowhere is this more obvious than in Google’s own public data tools. Try getting information about Google websites in any of these tools and you will notice that most of this information is nowhere to be found. You’ll be able to find information about any website on the Internet, but not Google’s own.

Examples of how Google blocks its own sites

Here are a few examples that we know of:

• Google Trends for Websites – Will not let you view traffic trends for Google.com or any of its country-specific versions, or for Youtube.com.
• Google Ad Planner (a.k.a. DoubleClick Ad Planner) – Will not let you view traffic or demographics data for Google sites such as Google.com and Youtube.com.
• Google’s list of the 1,000 most-visited websites – This list contains visitor and page view information for the top websites, but doesn’t include Google.com (or its country-specific versions) or Youtube.com.

There is a certain irony here; many of Google’s own tools are Google-free zones.

Just look at this:

The regular Google Trends, the one for search stats, doesn’t block you from viewing search trends for words such as “Google”, but apparently Google draws the line at site stats.

Why?

Turning to Google’s own help pages for its tools doesn’t enlighten us much. Here’s an example from Google’s Trends for Websites (emphasis below added by us):

Not all websites are included in Trends for Websites. The following types of websites may not appear in the tool:

• Websites with low traffic volume below our threshold
• Websites that don’t wish to be indexed by Google and have indicated their preference through a robots.txt exclusion file
• Websites that don’t adhere to our Quality Guidelines
• Other websites for miscellaneous reasons

It’s like there’s an elephant in the room and Google refuses to acknowledge it.

When TechCrunch asked Google about why it excluded its own sites from Trends for Websites a couple of years ago, the answer they got back was about as generic as it gets:

We have policy of not providing interim financial guidance, and have decided not to release Google numbers in accordance with that policy.

Google isn’t even being consistent about this

Since this is all apparently the result of Google policy, you’d expect this to be the same for all of Google’s sites.

But no.

Sites that do tend to work are Blogger.com (Blogspot.com) and Orkut.com. Perhaps Google considers this information less sensitive? We simply don’t know, but this is rather inconsistent behavior, isn’t it?

Ok, Google’s opting out. Can you?

Short answer, no.

The only way to opt out from having your website stats available in tools like Trends for Websites is to completely remove your site from Google’s index (via robots.txt), but who would want to do that?

So Google can opt out, but you can’t.

Conclusion

These tools from Google are great and highly useful, and we think it’s wonderful that Google has made them freely available. However, it is a bit odd that Google so consistently refuses to put its own sites on the same playing field as the rest of the web.

Google actively promotes openness. It encourages us to share our information, be as public as we can, and Google will help make that information accessible. So why is Google so willing to let us probe into the stats of other websites and services out there on the Internet, yet will not do the same for its own services? And in its own tools, at that.

It’s their choice, of course, but it seems to be very contrary to Google’s focus on openness.

The .com top-level domain has dominated the Internet pretty much from the start, and that’s still the case. But how strong is this dominance? After all, there are now approximately 200 million registered domain names, and less than half of those are .coms.

To find out what the current situation looks like for actual, popular websites, we’ve looked at this from two different perspectives:

• The top 10,000 websites in the world.
• The top 10,000 websites in the United States.

This article will show you the distribution of top-level domains (TLDs) among these top websites to show you how widely used .com is today, and how the other top-level domains are doing by comparison.

The .com TLD in the world vs. the US

We get two very different pictures depending on which set of websites we look at, although the pattern is similar.

But before we proceed, let’s get two definitions out of the way for those of you not fluent in domain speak:

• ccTLD: country code top-level domain, for example .de, .uk, .cn, etc. There are 240+ of these.
• gTLD: generic top-level domain, for example .com, .net, .org, etc.

Now that that has been cleared up, let’s get to the charts.

TLD distribution among top websites worldwide

So what does the TLD distribution look like for the top 10k websites in the world? Here it is, as a pie chart for your viewing pleasure. (Pacman, anyone?)

In the chart above, we only included TLDs with 1% or more of the worldwide top 10k. The rest were lumped together in the “other” category.

As you can see .com has a very dominant position, which was to be expected. Since ccTLDs tend to be country-specific, it will be hard for them to reach the very top worldwide.

A few points of interest for the worldwide top 10k sites:

• Not only does .com dominate with 61.8%, but the nearest TLD is .net with just 5.8%. That’s quite a difference.
• All gTLDs counted together make up 71.8%.
• The ccTLDs that do show up are a reflection of the large number of Internet users in those countries, such as Germany (.de), Russia (.ru), Japan (.jp), China (.cn), etc. One might also assume that these are nations with a high number of strong nation-specific website properties.

So what does the situation look like if we only study websites that are popular in the United States?

TLD distribution among top websites in the United States

We had a feeling that the difference between the worldwide and the US-only perspectives would be big, and we were right. If the worldwide chart showed .com as a Pacman ready to bite, in this one Pacman is gulping down the other TLDs like an afternoon snack.

Again, we only included TLDs that had 1% or more of the top 10k websites. The rest were put in the “other” category.

A few points of interest:

• Although the US does have its own ccTLD, .us, only 89 of the US top 10k sites are on .us domains. That’s less than 1%.
• The .com domain has 83.3% (versus 61.8% worldwide).
• All gTLDs counted together make up 95.7% (versus 71.8% worldwide).

What does this tell us?

What it tells us is that whichever way you slice it, .com is still king.

Perhaps we are all, and Americans in particular, by now so schooled to seeing and using .com domains for top websites that other TLDs may be seen with some ingrained level of suspicion.

Maybe something to keep in mind when you’re registering a domain for your next startup, at least if you’re aiming for an international audience.

Data sources:
Top 10k sites worldwide from Alexa.
Top 10k sites in the US from Quantcast.

Everyone knows that the really big and popular websites have a ton of visitors. But how many? How much does it take to reach the top?

We were curious about this ourselves. We’ve all seen the almost absurd numbers involved with the very top sites (like Facebook), but what about website number 50, or 100, or 500? What does it take for a website, in terms of visitors, to reach such a ranking?

A while back Google released a list of the 1,000 most-visited websites on the Internet, including estimates of how many worldwide visitors those websites have each month. We used this list as a basis for this little study.

Key findings

There are some very revealing patterns that emerge when you look at this many websites at once, but before we go ahead and show you those, here are a few key findings:

• To become a top 1,000 website you need at least 4.1 million visitors per month.
• To become a top 500 website you need at least 7.4 million visitors per month.
• To become a top 100 website you need at least 22 million visitors per month.
• To become a top 50 website you need at least 41 million visitors per month.
• To become a top 10 website you need at least 230 million visitors per month.
• To become the number 1 website in the world? Then you need more than 540 million visitors per month. (Good luck with that.)

Facebook is the site behind that last number.

And just to be clear, whenever we say “visitors” in this article, we mean unique visitors.

The visitor numbers for the top websites

To give you a good overview how the visitor numbers are distributed over the top websites, we have created two charts for you. One shows the top 100 websites, and one shows the top 1,000. Each chart shows the number of visitors these websites receive per month.

Here’s the first one, the top 100 websites:

As you can see, a clear pattern emerges, with a sharp increase in visitor numbers for the very top websites, and the curve flattening out more and more the longer down the list you go. The number of visitors websites receive is clearly very much a case of the famous “long tail”.

The long tail becomes even more obvious when you expend the scope to show the top 1,000 websites:

Above: At this scale columns just look messy, so we used a line graph.

The effect of the long tail

What the above charts show is very much an example of “the long tail” in terms website visitors.

If you sum up the number of visitors each website receives, you get very telling ratios when you compare the top 10, top 100 and top 1,000.

• The top 10 websites get 42% of the visitors to the top 100, and 21% of the visitors to the top 1,000.
• The top 100 gets 50% of the visitors to the top 1,000. I.e. the top 100 together get as many visitors as the following 900 websites counted together.

Visitor numbers are a bit special in that there of course will be be plenty of overlap between the websites. After all, we web users tend to visit more than one site. This overlap will be bigger the higher up in the rankings you go.

Final words

So there you have it. If you want to break into the top 100 websites, you’ll need at least 22 million visitors per month. If you “only” aim for the top 1,000, you’ll need at least 4.1 million visitors per month.

Gee, that can’t be so hard, can it…?

A note about the data:
This data comes from Google, which in turn uses both its own data and that of third parties to calculate these estimates (and they ARE estimates).

One interesting bit about this is that Google tends to exclude most of its own properties/sites from such lists, so the charts above did not include the numbers for Google Search and YouTube, to name a couple of prominent examples. We adjusted the “what it takes to be a top 10 site” number to take this into consideration, since we know that both Google Search and YouTube are top 10 sites. The charts, however, don’t include these sites since we don’t have comparable visitor numbers for them.

That said, even with a few site omissions from Google, the overall results should give you a good indication of what it takes, in terms of visitor numbers, to be counted among the top sites of the world.

Microsoft just announced that it has sold 150 million copies of Windows 7, and that the OS is selling at a pace of seven per second. Not bad for an OS that was launched just eight months ago.

At that pace, Windows 7 will rapidly be gaining market share. So how is it doing compared to its older siblings, Windows Vista and Windows XP?

Here’s a graph showing how the overall market shares for these three operating systems have changed over the last 12 months:

There are two amazing observations to take away from this chart:

• Windows 7 looks set to overtake Vista within one or at the most two months.
• Windows XP is still king of the hill with a dominating 58% share of the OS market.

So, Windows 7 will have overtaken Vista after being on the market for less than a year, which is impressive, but it will clearly take much longer before it replaces Windows XP as the dominant version of Windows. Keep in mind that Windows XP is already ancient; it was launched in October 2001 and apparently still refuses to let go.

Back in February we showed that Windows 7 was generating a public interest far greater than Vista ever was able to do. We also asked the question: “How long will it take for Windows 7 to overtake Vista in actual market share?”

Now we know.

Data source: The OS market share numbers come from StatCounter and are based on visitor stats from more than three million websites.

That Twitter has been expanding well outside the borders of the United States is well known. It’s becoming a global social media phenomenon and much of its growth is currently coming from this international expansion. We thought it would be interesting to investigate where Twitter is growing the fastest right now.

By looking at traffic data to Twitter.com, we’ve identified several countries where Twitter is currently growing rapidly (this year). These are not necessarily countries that have a ton of Twitter users yet, but rather places where the traffic curve is pointing sharply upwards.

(The traffic estimates in this article are from Google’s Trends for Websites service.)

Latin America coming strong

One interesting thing we noticed when going through the traffic data is that Twitter seems to be growing rapidly in several Latin American countries (links below lead to each country’s traffic page on Google Trends for Websites):

• Argentina
• Brazil
• Colombia
• Mexico
• Venezuela

Above: The graphs show traffic to Twitter.com through 2009 until now.

These are not small countries. Together, they have almost 150 million Internet users that Twitter can potentially tap into. For perspective, there are 234 million Internet users in the United States.

Strong growth in Asia

A second area where Twitter seems to be getting a boost is in Asia, especially in East Asia, accounting for three out of the four countries we list here below.

• India
• Japan
• South Korea
• Taiwan

Above: The graphs show traffic to Twitter.com through 2009 until now.

Once again, a lot of Internet users to go for. Together, these four countries have 230 million Internet users. (An interesting aside: Japan actually has more Internet users than India, in spite of India’s huge overall population.)

That is roughly the equivalent of the number of Internet users in the United States.

Twitter is also getting increasing amounts of traffic from other parts of Asia, such as Thailand, the Philippines, Malaysia, Indonesia, and so on.

Other countries on the rise

Twitter’s expansion does not end with the countries we’ve listed above. Here are a few other countries where traffic to Twitter.com has increased a lot so far this year:

• Italy
• Russia
• Spain

Above: The graphs show traffic to Twitter.com through 2009 until now.

These three countries have 104 million Internet users between them (Russia alone has 45 million).

And there are many other countries where Twitter is growing (most European countries, for example). We simply cherry picked the ones where this year’s traffic increase to Twitter.com is extra sharp.

Huge potential

The countries we’ve listed in this article have a total of 483 million Internet users. That’s a lot of room for Twitter to keep growing. And note that we have only highlighted the most rapidly growing countries. It’s not like Twitter has no presence or growth elsewhere. (And Twitter is clearly growing fast.)

Growing internationally is important if Twitter wants to become the pulse of the web. The United States may be big, but it’s worth remembering that the country only accounts for 13% of the world’s Internet users.

A final word on language

Another fascinating aspect here is language. So far, the dominant language on Twitter has been English. However, with the exception of India, none of these countries are big on using English. This means that the number of non-English tweets will increase significantly in the coming year(s).

This is of course an inevitable consequence of Twitter going global, but it will be interesting to see how Twitter and its users adapt.

Data sources: Google Trends for Websites for traffic data, Internet World Stats for Internet user numbers.

At the scale that Facebook operates, a lot of traditional approaches to serving web content break down or simply aren’t practical. The challenge for Facebook’s engineers has been to keep the site up and running smoothly in spite of handling close to half a billion active users. This article takes a look at some of the software and techniques they use to accomplish that.

Facebook’s scaling challenge

Before we get into the details, here are a few factoids to give you an idea of the scaling challenge that Facebook has to deal with:

• Facebook serves 570 billion page views per month (according to Google Ad Planner).
• There are more photos on Facebook than all other photo sites combined (including sites like Flickr).
• More than 3 billion photos are uploaded every month.
• Facebook’s systems serve 1.2 million photos per second. This doesn’t include the images served by Facebook’s CDN.
• More than 25 billion pieces of content (status updates, comments, etc) are shared every month.
• Facebook has more than 30,000 servers (and this number is from last year!)

Software that helps Facebook scale

In some ways Facebook is still a LAMP site (kind of), but it has had to change and extend its operation to incorporate a lot of other elements and services, and modify the approach to existing ones.

For example:

• Facebook still uses PHP, but it has built a compiler for it so it can be turned into native code on its web servers, thus boosting performance.
• Facebook uses Linux, but has optimized it for its own purposes (especially in terms of network throughput).
• Facebook uses MySQL, but primarily as a key-value persistent storage, moving joins and logic onto the web servers since optimizations are easier to perform there (on the “other side” of the Memcached layer).

Then there are the custom-written systems, like Haystack, a highly scalable object store used to serve Facebook’s immense amount of photos, or Scribe, a logging system that can operate at the scale of Facebook (which is far from trivial).

But enough of that. Let’s present (some of) the software that Facebook uses to provide us all with the world’s largest social network site.

Memcached

Memcached is by now one of the most famous pieces of software on the internet. It’s a distributed memory caching system which Facebook (and a ton of other sites) use as a caching layer between the web servers and MySQL servers (since database access is relatively slow). Through the years, Facebook has made a ton of optimizations to Memcached and the surrounding software (like optimizing the network stack).

Facebook runs thousands of Memcached servers with tens of terabytes of cached data at any one point in time. It is likely the world’s largest Memcached installation.

HipHop for PHP

PHP, being a scripting language, is relatively slow when compared to code that runs natively on a server. HipHop converts PHP into C++ code which can then be compiled for better performance. This has allowed Facebook to get much more out of its web servers since Facebook relies heavily on PHP to serve content.

A small team of engineers (initially just three of them) at Facebook spent 18 months developing HipHop, and it is now live in production.

Haystack

Haystack is Facebook’s high-performance photo storage/retrieval system (strictly speaking, Haystack is an object store, so it doesn’t necessarily have to store photos). It has a ton of work to do; there are more than 20 billion uploaded photos on Facebook, and each one is saved in four different resolutions, resulting in more than 80 billion photos.

And it’s not just about being able to handle billions of photos, performance is critical. As we mentioned previously, Facebook serves around 1.2 million photos per second, a number which doesn’t include images served by Facebook’s CDN. That’s a staggering number.

BigPipe

BigPipe is a dynamic web page serving system that Facebook has developed. Facebook uses it to serve each web page in sections (called “pagelets”) for optimal performance.

For example, the chat window is retrieved separately, the news feed is retrieved separately, and so on. These pagelets can be retrieved in parallel, which is where the performance gain comes in, and it also gives users a site that works even if some part of it would be deactivated or broken.

Cassandra

Cassandra is a distributed storage system with no single point of failure. It’s one of the poster children for the NoSQL movement and has been made open source (it’s even become an Apache project). Facebook uses it for its Inbox search.

Other than Facebook, a number of other services use it, for example Digg. We’re even considering some uses for it here at Pingdom.

Scribe

Scribe is a flexible logging system that Facebook uses for a multitude of purposes internally. It’s been built to be able to handle logging at the scale of Facebook, and automatically handles new logging categories as they show up (Facebook has hundreds).

Hadoop and Hive

Hadoop is an open source map-reduce implementation that makes it possible to perform calculations on massive amounts of data. Facebook uses this for data analysis (and as we all know, Facebook has massive amounts of data). Hive originated from within Facebook, and makes it possible to use SQL queries against Hadoop, making it easier for non-programmers to use.

Both Hadoop and Hive are open source (Apache projects) and are used by a number of big services, for example Yahoo and Twitter.

Thrift

Facebook uses several different languages for its different services. PHP is used for the front-end, Erlang is used for Chat, Java and C++ are also used in several places (and perhaps other languages as well). Thrift is an internally developed cross-language framework that ties all of these different languages together, making it possible for them to talk to each other. This has made it much easier for Facebook to keep up its cross-language development.

Facebook has made Thrift open source and support for even more languages has been added.

Varnish

Varnish is an HTTP accelerator which can act as a load balancer and also cache content which can then be served lightning-fast.

Facebook uses Varnish to serve photos and profile pictures, handling billions of requests every day. Like almost everything Facebook uses, Varnish is open source.

Other things that help Facebook run smoothly

We have mentioned some of the software that makes up Facebook’s system(s) and helps the service scale properly. But handling such a large system is a complex task, so we thought we would list a few more things that Facebook does to keep its service running smoothly.

Gradual releases and dark launches

Facebook has a system they called Gatekeeper that lets them run different code for different sets of users (it basically introduces different conditions in the code base). This lets Facebook do gradual releases of new features, A/B testing, activate certain features only for Facebook employees, etc.

Gatekeeper also lets Facebook do something called “dark launches”, which is to activate elements of a certain feature behind the scenes before it goes live (without users noticing since there will be no corresponding UI elements). This acts as a real-world stress test and helps expose bottlenecks and other problem areas before a feature is officially launched. Dark launches are usually done two weeks before the actual launch.

Profiling of the live system

Facebook carefully monitors its systems (something we here at Pingdom of course approve of), and interestingly enough it also monitors the performance of every single PHP function in the live production environment. This profiling of the live PHP environment is done using an open source tool called XHProf.

Gradual feature disabling for added performance

If Facebook runs into performance issues, there are a large number of levers that let them gradually disable less important features to boost performance of Facebook’s core features.

The things we didn’t mention

We didn’t go much into the hardware side in this article, but of course that is also an important aspect when it comes to scalability. For example, like many other big sites, Facebook uses a CDN to help serve static content. And then of course there is the huge data center Facebook is building in Oregon to help it scale out with even more servers.

And aside from what we have already mentioned, there is of course a ton of other software involved. However, we hope we were able to highlight some of the more interesting choices Facebook has made.

Facebook’s love affair with open source

We can’t complete this article without mentioning how much Facebook likes open source. Or perhaps we should say, “loves”.

Not only is Facebook using (and contributing to) open source software such as Linux, Memcached, MySQL, Hadoop, and many others, it has also made much of its internally developed software available as open source.

Examples of open source projects that originated from inside Facebook include HipHop, Cassandra, Thrift and Scribe. Facebook has also open-sourced Tornado, a high-performance web server framework developed by the team behind FriendFeed (which Facebook bought in August 2009).

(A list of open source software that Facebook is involved with can be found on Facebook’s Open Source page.)

More scaling challenges to come

Facebook has been growing at an incredible pace. Its user base is increasing almost exponentially and is now close to half a billion active users, and who knows what it will be by the end of the year. The site seems to be growing with about 100 million users every six months or so.

Facebook even has a dedicated “growth team” that constantly tries to figure out how to make people use and interact with the site even more.

This rapid growth means that Facebook will keep running into various performance bottlenecks as it’s challenged by more and more page views, searches, uploaded images, status messages, and all the other ways that Facebook users interact with the site and each other.

But this is just a fact of life for a service like Facebook. Facebook’s engineers will keep iterating and coming up with new ways to scale (it’s not just about adding more servers). For example, Facebook’s photo storage system has already been completely rewritten several times as the site has grown.

So, we’ll see what the engineers at Facebook come up with next. We bet it’s something interesting. After all, they are scaling a mountain that most of us can only dream of; a site with more users than most countries. When you do that, you better get creative.

Data sources: Various presentations by Facebook engineers, as well as the always informative Facebook engineering blog.