What shape is the internet (continued)?

@ironick and I have been arguing about the shape of the internet since my September 2010 post on this subject. Over the past few days, we have returned to this topic on Twitter. Nick has captured the latest tweets in his Storify piece The Shape of the Web - Database Wars Redux.

The argument was triggered by @djbressler's observation that some new browsers (including an experimental build of Chrome) were hiding the URL from the user. This is a reflection of the fact that users increasingly type "Amazon" into the browser rather than "amazon.com" let alone "http://www.amazon.com". Presumably, hiding the URL will further encourage this trend.

Google and other search engines appear to benefit from this in two ways. Firstly, it increases the already heavy dependence of the ordinary internet user on the search engine. And secondly, every time an internet user navigates via search rather than via URL or hyperlink, the search engine gets another opportunity to present some advertising, as well as collecting more information about that user.

Obviously, Google itself depends on URLs and hyperlinks. As Nick points out, Google still relies on links to construct its index, and still uses a version of the original PageRank algorithm to influence what you see when you search for a given term. But indexing and search ranking are only loosely coupled to one another.

And nowadays, the search order is not solely determined by PageRank. Instead, the search order is increasingly influenced by browsing behaviour - of others as well as our own. If you ignore the first two items, click briefly on the third item, and then immediately return to Google to look at the fourth item, Google may conclude that the first three items weren't very relevant to you. In other words, this counts as a "vote" against those items.

Meanwhile, Google only had exclusive rights to the original PageRank patent (which belongs to Stanford University) until 2011.

Obviously Google is not completely open about these algorithms, because it is perpetually at war with SEO and spammers who want to get some commercial advantage by "gaming" the system. So there is a degree of speculation involved in working out what exactly Google is up to. Sometimes Google merely seems to appeal to the lowest common denominator, as David Auerbach suggests in his review of Metafilter search results ("Deranked"). However, it is beyond speculation that Google's behaviour has become increasingly sophisticated over the past decade, and that what we see is increasingly "personalized".

Nick accuses me of "confusing the use of behavior IN the ranking algorithm itself with using behavior to verify the quality of the algorithm". However, there is some evidence that Google initially trials new factors in parallel with the existing algorithm, before integrating these factors into the algorithm itself. (See for example, Google Panda.) In any case, the total behaviour of Google can be thought of in terms of the collective intelligence of human brains AND algorithmic software, and it may not be possible for outsider observers to be exactly sure where the boundary lies at any point in time. (We can detect "momentum", but not "position".)

Obviously URLs are not going to disappear entirely. For my part, I have always made an effort to use links and bookmarks rather than pander to the commercial interests and cognitive distortion of search engines. I don't think this undermines my general point - that the Internet-in-use (based on majority habits) is taking on a different shape. Obviously it is still possible to use the Internet in a disciplined and self-conscious manner, which Nick (always) and I (sometimes) practise, but the fact that this requires effort and intelligence makes it likely that it will never become mainstream.

In the long-term, Google may face a paradox. If people stop using URLs, then Google's ability to index and rank pages across the internet might possibly be compromised. But I'm sure that the clever people at Google have thought of this paradox, and already have a cunning plan.

Meanwhile, the internet (as experienced by ordinary users) is gradually becoming less web-shaped and more star-shaped, with your favourite search engine or social network at the centre. (Please note the word "gradually".)

---

Sources

David Auerbach, Deranked - Why has Google forsaken MetaFilter? (Slate May 2014)
Bill Slawski, The New PageRank, Same as the Old PageRank? (March 2012)
Daniel Sour, It Knows (LRB October 2011)


Related posts

What shape is the internet (September 2010)
What shape is your intranet (May 2014)

Updated  17 May 2014