A funny thing happened on the way to Web markteting in 2016: the world of SEO ballooned under your inattentive noses. Most of you are still trapped in the “mobile SEO” rut and you cannot see over the rim that there is a lot more search engine optimization going on than just mobile. We haven’t seen this many search environments since the 1990s, but today’s search environments are each larger than the Searchable Web Ecosystem was in 1998. I cannot begin to describe just how much larger and more varied today’s searchscape is compared to what we had 18 years ago.
It seems like it was just yesterday that Altavista was confirming at a search conference that they had implemented a new doorway page filter and stopped the massive inflow of doorway page submissions to a round of enthusiastic applause. Web spam had been defeated and real marketers could go on to compete in a real Web marketing sphere. Back then it sounded so convincing.
In fact, even as Web spam came back to haunt the search engines, so did the technologies that Altavista developed to make search better (including providing answers to questions, phrase-based indexing, and real time spam detection). When you read the article “AltaVista Debuts Search Features” in that archived newsletter from November 1998, pay special attention to this paragraph:
Monier said the phrase detection has been built through a completely automated process, which takes advantage of linguistic patterns to determine what is a phrase, and what isn’t.
Oh, doesn’t that sound familiar to anyone?
For those of us who were studying search engines 18 years ago it should come as no surprise that much of what Google is doing today was done by Altavista back then. Of course, it’s a bit different now because Google’s engineers have (hopefully) learned from 18 years of search engineering successes and failures.
The scope of what Bing and Google do today is at least 10 times larger, maybe closer to 50 times larger, than what search engines like Altavista and Inktomi did back then. Sure, you could say that their scopes are even larger based on the sizes of indexes, but as Danny Sullivan noted during the Index Wars, size doesn’t seem to matter if quality is not keeping up with the index.
For those who are keeping score, Google’s search results are derived from an index that is slightly larger than 100 billion URLs1. They say they crawl 60 trillion URLs but they have managed to boil those trillions down to a fraction of their crawl. I’ll take that as a sign that the quality of search results has improved somewhat in the last 18 years, even if Google’s index is more than 200 times the size of Inktomi’s 500 million URLs (which was about where people stopped counting or caring).
This trip down memory lane has a purpose. I want you to think about search engines other than Bing and Google. It’s important that you remember there are search engines other than Bing and Google as you go forward. You just don’t normally think of them as search engines.
I’m not talking about Ask, AOL, or your favorite meta search engine. I’m talking about the most popular, most important search engines on the Web.
They all have to solve similar problems dealing with relevance, speed, and filtering out crud (spam, duplicate content, and irrelevant autogenerated gibberish).
Let’s take a look at the major search engines of 2016 and 2017. You will recognize all of them but you won’t agree they are all search engines. For those of you who want to argue with me, that is why you are missing multiple search optimization opportunities.