|
|
|
Search Engine Optimisation (SEO) History
The following article provides an excellent brief overview of the
history of the commercial internet and forms the introduction to a
series of articles on SEO. More coming soon.
A Brief History of the Search Engine
The World Wide Web was born in November, 1990, with the launch of the
first Web server (and Web page) hosted at the CERN research facility in
Switzerland. Not surprisingly, the purpose of the first Web page was to
describe the World Wide Web project. At the time, no search engine was
needed—you could literally read the entire contents of the World Wide
Web in less than an hour.
By early 1993, the stage was set for the Web explosion. In February of
that year, the first (alpha) release of the NCSA Mosaic graphical
browser provided a client application that, by the end of the year, was
available on all major desktop computing platforms. The Netscape
browser, based on Mosaic, was released in 1994. By this time, dial-up
Internet access had become readily available and was cheap. The Web was
taking off!
The Early Days of Web Search.
Even though the combination of cheap dial-up access and the Mosaic
browser had made the Web semi-popular, there was still no way to search
the growing collection of hypertext documents available online. Most
Web pages were basically collections of links, and a popular pastime of
Web users was to share their bookmark files.
This isn't to say that attempts weren't made to bring order to the
swiftly growing chaos. The first automated Web crawler, or robot, was
the World Wide Web Wanderer created by MIT student Mathew Gray. This
crawler did little more than collect URLs, and was largely seen as a
nuisance by the operators of Web servers. Martjin Koster created the
first Web directory, ALIWeb, in late 1993, but it, like the Wanderer,
met with limited success.
In February 1993, six Stanford graduate students began work on a
research project called Architext, using word relationships to search
collections of documents. By the middle of that year, their software
was available for site search. More robots had appeared on the scene by
late 1993, but it wasn't until early 1994 that searching really came
into its own.
The Great Search Engine Explosion
1994 was a big year in the history of Web search. The first
hierarchical directory, Galaxy, was launched in January and, in April,
Stanford students David Filo and Jerry Yang created Yet Another
Hierarchical Officious Oracle, better known as Yahoo!.
During that same month, Brian Pinkerton at the University of Washington
released WebCrawler. This, the first true Web search engine, indexed
the entire contents of Web pages, where previous crawlers had indexed
little more than page titles, headings, and URLs. Lycos was launched a
few months later.
By the end of 1995, nearly a dozen major search engines were online.
Names like MetaCrawler (the first metasearch engine), Magellan,
Infoseek, and Excite (born out of the Architext project) were released
into cyberspace throughout the year. AltaVista arrived on the scene in
December with a stunningly large database and many advanced features,
and Inktomi debuted the following year.
Over the next few years, new search engines would appear every few
months, but many of these differed only slightly from their
competitors. Yet the occasional handy innovation would find its way
into practical use. Here are a few of the most successful ideas from
that time:
- GoTo (now Overture) introduced the concept of pay-per-click (PPC)
listings in 1997. Instead of ranking sites based on some arcane
formula, GoTo allowed open bidding for keywords, with the top position
going to the highest bidder. All major search portals now rely on PPC
listings for the bulk of their revenues.
- Metasearch engines, which combine results from several other
search engines, proliferated for a time, driven by the rise of
pay-per-click systems and the inconsistency of results among the major
search engines. Today, new metasearch engines are rarely if ever seen,
but those that remain possess a loyal following. The current crop of
metasearch engines display mostly pay-per-click listings.
- The Mining Company (now About) launched in February 1997, using
human experts to create a more exclusive directory. Many topic-specific
(vertical) directories and resource sites have been created since, but
About remains a leading resource.
- DirectHit introduced the concept of user feedback in 1998,
allocating a higher ranking to sites whose listings were clicked by
users. DirectHit's data influenced the search results on many portals
for a long time, but, because of the system's susceptibility to
manipulation, none of today's search portals openly use this form of
feedback. DirectHit was later acquired by Ask Jeeves (now Ask), and
user behavior may well be factored into the Ask/Teoma search results we
see today.
- Pay-to-play was introduced, as search engines and directories
sought to capitalize on the value of their editorial listings. The
LookSmart and Yahoo! directories began to charge fees for the review
and inclusion of business Websites. Inktomi launched "paid inclusion"
and "trusted feed," allowing site owners to ensure their inclusion
(subject to editorial standards) in the Inktomi search engine.
- The examination of linking relationships between pages began in
earnest, with AltaVista and other search engines adding "link
popularity" to their ranking algorithms. At Stanford University, a
research project created the Backrub search engine, which took a novel
approach to ranking Web pages.
Google Dominates, the Field Narrows
The Backrub search engine eventually found its way into the public
consciousness as Google. By the time the search engine was officially
launched as Google in September 1998, it had already become a very
popular player.
The development of search engines since that time has been heavily
influenced by Google's rise to dominance. More than any other search
portal, Google has focused on the user experience and quality of search
results. Even at the time of its launch, Google offered users several
major improvements, some of which had nothing to do with the search
results offered.
One of the most appealing aspects of Google was its ultra-simple user
interface. Advertising was conspicuously absent from Google's
homepage—a great advantage in a market whose key players typically
adorned their pages with multiple banners—and the portal took only a
few seconds to load even on a slow dial-up connection. Users had the
option to search normally, but a second option, called "I'm Feeling
Lucky," took users directly to the page that ranked at the top of the
results for their search.
Like its homepage, Google's search results took little time to appear
and carried no advertising. By the time Google began to show a few paid
listings through the AdWords service in late 2000, users didn't mind:
Google had successfully established itself as the leading search portal
and, unlike many other search engines, it didn't attempt to hide paid
advertising among regular Web search results.
Many other search portals recognized the superiority of Google's search
results, and the loyalty that quality generated. AOL and Yahoo! made
arrangements to display Google's results on their own pages, as did
many minor search portals. By the end of 2003, it was estimated that
three-quarters of all Web searches returned Google-powered results.
Within a few years, the near-monopoly that Google achieved in 2003 will
be recognized as a high water mark, but the development of this search
engine is by no means finished.
The years 2001–2003 saw a series of acquisitions that rapidly
consolidated the search industry into a handful of major players.
Yahoo! acquired the Inktomi search engine in March 2003; Overture
acquired AltaVista and AllTheWeb a month later; Yahoo! announced the
acquisition of Overture in August 2003.
In 2004, a new balance of power took shape:
- Yahoo! released its own search engine powered by a fusion
of the AltaVista, Inktomi, and AllTheWeb technology they acquired in
2003. Yahoo! stopped returning Google search results in January 2004.
- Google's AdWords and AdSense systems, which deliver
pay-per-click listings to search portals and Websites respectively,
grew dramatically. Google filed for an initial public offering (IPO).
- The popularity of the Ask search portal, powered by the
innovative Teoma search engine, steadily increased. Like most portals
that Yahoo! doesn't own, Ask uses Google's AdWords for paid listings.
- The 800-lb gorilla of the computing world, Microsoft,
announced plans for its own search engine, releasing beta versions for
public use in January and June of 2004, and formally launching the
service in February 2005. Microsoft now offers MSN search results on
the MSN portal.
Source, D.Thies, The Search Engine Marketing Kit
The original article can be viewed at the very useful web developers recourse Sitepoint here
[ printer friendly version ] [ top of page ]
|
|
