The Power of Twitter

Let's suppose I want to find an intelligent review of a film.

If I just put the name of the film into Google, I will get endless repetitions of the synopsis, together with details of cinemas showing the film, or places to buy/download.

In my post You don't have to be smart to search here ... but it helps (Nov 2008), I outlined one possible trick. If you put the name of the film together with a random cultural icon (my example was Lacan), you will get reviews of the film that name-drop the icon. That immediately filters out all the standard cinema listings. However, you might need to try a number of different cultural icons until you strike lucky.

A second option is to subscribe to good magazines. When I watched the film Anomalisa, I didn't immediately make the connection with Schopenhauer. The connection was made for me by a fascinating review by Zadie Smith in the New York Review of Books.

Zadie Smith on Anomalisa, Schopenhauer, human desire and despair https://t.co/JQFwM0P5Qz

— NY Review of Books (@nybooks) February 19, 2016

Once you know that such a connection exists, you can use Google to find it. But Google won't make that connection for you - unless sufficient numbers of other people have already made that connection.

So here's a third option. Twitter allows you to have a list of intelligent film critics, and intelligent magazines containing intelligent film reviews. Either you decide for yourself what counts as intelligent, or you adopt someone else's list. Then you can search through the list for seriously intelligent reviews of the latest film. You can't do anything quite like this with Google.


When you search for something, Google can give you page after page of practically identical material - for example, hundreds of newspapers all repeating the same press release. What one really wants is a search engine that works out which page represents the original source, which pages represent replications with no added content or value, and which pages offer additional commentary and interpretation. It is possible that Twitter, with its conversational structure, may be closer to providing this kind of navigation. But only if the platform can achieve reasonable commercial viability without being polluted.

The Force of Goole

When people talk about Internet Binging, they aren't talking about using the world's fourth most popular internet search engine. According to @ruskin147's BBC Radio Four documentary The Force of Google this evening, people don't even use the generic phrase "searching the internet". They use the word "Google". I think I heard someone say that the word is now more popular than the word "eggs".

Rory discussed several ways that hard-boiled Google poaches Internet business, while scrambling our brains.

1.  Business is dependent on the caprice of Google ranking. Rory talks to the owner of a fly fishing company, which gets a significant proportion of his business via Google. When Google changed its algorithm in 2013, his webpage dropped from page one to page seven - almost equivalent to a commercial death penalty. Then inexplicably it climbed back again - the death penalty reprieved. Readers with long memories will remember the story of BMW (Feb 2006), which was banished from Google for three days in 2006.

 

2. In trying to be as helpful as possible to searchers, Google sometimes fails to respect the interests of other information providers. For example, if you search for hotels in Bury, you get Google's automatically curated list before you get lists from rival platforms such as TripAdvisor and Yelp. 

3. In the past, there has been some evidence that Google is biased towards controversial new technologies, perhaps because the technology vendors spend more on advertising than the technology sceptics. I have noted this apparent bias in relation to Biometrics (Nov 2003) and RFID (Nov 2005). Google now seems to have made some progress on this issue - Rory looked up "fracking" and got a more even-handed view from Google than from Bing.

4. Even without any obvious commercial or political agenda on Google's part, it is easy to see how Google's results could appear to show a lack of balance. Note for example the recent controversy about Unprofessional Hair. There have also been suggestions that Google page ranking could influence the public perception of politicians and thus sway elections.

5. One of the most dangerous aspects of the Google phenomenon is the widespread illusion that Google gives you Objective Truth. Rory talks to Ben Gomes, who is described as Google's Guru of Search, who talks about the Quest for the Perfect Search.

"The perfect search is giving you what you were looking for. Not just the words you typed - but what you were actually looking for."

The programme gave the impression that Google is converging on the Perfect Search. Rory himself says he generally finds what he is looking for. My own experience is that it sometimes requires a fair amount of ingenuity to find stuff, especially interesting and original stuff. See my posts You don't have to be smart to search here ... but it helps (Nov 2008) and Thinking with the Majority (March 2009). See also The Power of Twitter (April 2016).

---

Wondering about the deliberate spelling mistake in the title of this post? I wanted to pay tribute to a listing from @brightonargus.

According to the Brighton Argus I've made a documentary about the power of a town in East Yorkshire.... pic.twitter.com/UGhvcO2Ovv

— Rory Cellan-Jones (@ruskin147) April 26, 2016

Which reminded me of the original Argus Panoptes, the giant who would be the mythical ancestor of Google. And also the ARGUS-IS system, a secret rival to Google's Street View.  Even Argus may have flawed vision sometimes.

Wikipedia: Argus Panoptes, ARGUS-IS.

---

Leigh Alexander, Do Google's 'unprofessional hair' results show it is racist? (Guardian 8 April 2016)

Rory Cellan-Jones, Six searches that show the power of Google (BBC 26 April 2016)

Konrad Krawczyk, Google is easily the most popular search engine, but have you heard who’s in second? (Digital Trends, 3 July 2014)

What shape is the internet (continued)?

@ironick and I have been arguing about the shape of the internet since my September 2010 post on this subject. Over the past few days, we have returned to this topic on Twitter. Nick has captured the latest tweets in his Storify piece The Shape of the Web - Database Wars Redux.

The argument was triggered by @djbressler's observation that some new browsers (including an experimental build of Chrome) were hiding the URL from the user. This is a reflection of the fact that users increasingly type "Amazon" into the browser rather than "amazon.com" let alone "http://www.amazon.com". Presumably, hiding the URL will further encourage this trend.

Google and other search engines appear to benefit from this in two ways. Firstly, it increases the already heavy dependence of the ordinary internet user on the search engine. And secondly, every time an internet user navigates via search rather than via URL or hyperlink, the search engine gets another opportunity to present some advertising, as well as collecting more information about that user.

Obviously, Google itself depends on URLs and hyperlinks. As Nick points out, Google still relies on links to construct its index, and still uses a version of the original PageRank algorithm to influence what you see when you search for a given term. But indexing and search ranking are only loosely coupled to one another.

And nowadays, the search order is not solely determined by PageRank. Instead, the search order is increasingly influenced by browsing behaviour - of others as well as our own. If you ignore the first two items, click briefly on the third item, and then immediately return to Google to look at the fourth item, Google may conclude that the first three items weren't very relevant to you. In other words, this counts as a "vote" against those items.

Meanwhile, Google only had exclusive rights to the original PageRank patent (which belongs to Stanford University) until 2011.

Obviously Google is not completely open about these algorithms, because it is perpetually at war with SEO and spammers who want to get some commercial advantage by "gaming" the system. So there is a degree of speculation involved in working out what exactly Google is up to. Sometimes Google merely seems to appeal to the lowest common denominator, as David Auerbach suggests in his review of Metafilter search results ("Deranked"). However, it is beyond speculation that Google's behaviour has become increasingly sophisticated over the past decade, and that what we see is increasingly "personalized".

Nick accuses me of "confusing the use of behavior IN the ranking algorithm itself with using behavior to verify the quality of the algorithm". However, there is some evidence that Google initially trials new factors in parallel with the existing algorithm, before integrating these factors into the algorithm itself. (See for example, Google Panda.) In any case, the total behaviour of Google can be thought of in terms of the collective intelligence of human brains AND algorithmic software, and it may not be possible for outsider observers to be exactly sure where the boundary lies at any point in time. (We can detect "momentum", but not "position".)

Obviously URLs are not going to disappear entirely. For my part, I have always made an effort to use links and bookmarks rather than pander to the commercial interests and cognitive distortion of search engines. I don't think this undermines my general point - that the Internet-in-use (based on majority habits) is taking on a different shape. Obviously it is still possible to use the Internet in a disciplined and self-conscious manner, which Nick (always) and I (sometimes) practise, but the fact that this requires effort and intelligence makes it likely that it will never become mainstream.

In the long-term, Google may face a paradox. If people stop using URLs, then Google's ability to index and rank pages across the internet might possibly be compromised. But I'm sure that the clever people at Google have thought of this paradox, and already have a cunning plan.

Meanwhile, the internet (as experienced by ordinary users) is gradually becoming less web-shaped and more star-shaped, with your favourite search engine or social network at the centre. (Please note the word "gradually".)

---

Sources

David Auerbach, Deranked - Why has Google forsaken MetaFilter? (Slate May 2014)
Bill Slawski, The New PageRank, Same as the Old PageRank? (March 2012)
Daniel Sour, It Knows (LRB October 2011)


Related posts

What shape is the internet (September 2010)
What shape is your intranet (May 2014)

Updated  17 May 2014

What shape is your Intranet?

@djbressler tells us he is work­ing on a thought-piece about the bifur­ca­tion of the intranets from the Inter­net. In the meantime, in a post called Burying the URL (May 2014), he comments on an experimental build of Chrome, which continues the trend of hiding the URL and encouraging people to use search instead. Obviously it benefits Google when people rely exclusively on search. But it's not just Google's Chrome that is doing this; Firefox and Mobile Safari are also going in this direction.

In my post What shape is the internet (September 2010), I said that shifting the emphasis from URL hotlinks to search undermines the idea of the internet's being web-shaped. This point is also made in a post by @apike, referenced by David and also called Burying the URL (April 2014).

URLs are the essence. They make hypertext hyper. The term “web” is no accident – it refers to this explicitly.

See also an excellent Twitter debate following @apike's tweet.


When David talks about bifurcation, he means that "enter­prise IT is diverg­ing enter­prise tech­nol­ogy from con­sumer tech­nol­ogy in a way that’s cre­at­ing two irrec­on­cil­able branches of tech­nol­ogy". He observes that most company intranets have a pretty lousy search facility.

But most company intranets have pretty lousy cross-linking as well. They are mostly just pdf graveyards stuffed with documents of indeterminate pedigree, which people are often reluctant to waste time searching (even if the search facility were better) because they don't expect to find anything of value.

Actually, you can't always find what you are looking for on the Internet either, and that has a lot to do with the limitations of search, but there are enough amusing distractions to conceal this fact. Surely we don't want our company intranets to copy the internet too closely?

And remember that the data revealing Enron's problems were cheerfully displayed on the Enron website. But nobody important had bothered to look at these documents properly. (Actually, a bunch of students had analysed them years previously and concluded that Enron was bankrupt. They probably got low marks for that assignment!)

There is an increasingly common belief that the tech­nol­ogy used inside com­pa­nies should work the same way as outside, should provide the same "affordance". This is not Bring Your Own Device but Bring Your Own Paradigm Expectations. I guess I should work on a thought piece about this.

---

Related posts

What shape is the internet (September 2010)
Bring Your Own Expectations (May 2014)

See also Steven Poole, The pdf graveyards can only expect an increase in their undead populations (Guardian 9 May 2014)

Using Analytics Correctly

@gcharlton quotes a survey from @dbdsearch claiming that 80% of online retailers are using @GoogleAnalytics incorrectly (October 2012), via @FreshNick @hayden30.


Clearly Google wants online retailers to use all the features of the Google Analytics platform, which entails integrating with various other Google products and services (e.g. Google Adwords) as well as implementing all the necessary tracking codes and cookies according to Google's requirements. Any online retailer that fails to conform to Google's requirements is deemed to be using the platform incorrectly.

But what does "incorrectly" mean? Not doing what Google thinks you should be doing? Since when has Google been the ultimate arbiter of correct action?

We have been here many times before. There is often a significant gap between the designed product (how its designers expect it to be used) and the product-in-use (what the users actually do with the product). A designed product may have a number of sophisticated features that most users never get around to using, perhaps never actually need. On the other hand, the users trying to do a real job of work often display remarkable ingenuity in getting the stupid product to do something much more interesting than it was designed to do.

And sometimes there is a considerable delay until users discover the more sophisticated features. To cite a historical example, most early users of Lotus Notes used it as a substitute for technologies they already had, before they started to appreciate what it had really been designed for.

So there may be many ways people could learn to use Google Analytics better. As @haydens30 says, "there are basic best practice things that a lot of sites don't do - these are easy wins for any consultant".

And there may be many ways Google itself could make Analytics better and easier to use. In announcing some UI improvements yesterday, Nikhil Roy of the Google Analytics Team said "We hope you find these improvements useful and always feel free to let us know how we can make Analytics even more usable for you to get the information you need to take action faster."

What, we have to tell them? Don't they already know?
 

Google and Received Opinion

Brilliant satire from @newsbiscuit : New Google only searches for sites that match your preconceived opinions.



I have long complained that Google provides a systematically distorted way of finding out what is going on, and encourages what A.A. Milne called Thinking with the Majority. This is because Google's page ranking algorithms are basically designed for people who want to ask the same questions as everyone else, and get the same answers. Consequently, Google helps to amplify the circulation of Received Opinion.

The distortion is further amplified by massive duplication of material from a common source. If you search for a topical story, you will often find hundreds of popular websites repeating exactly the same version of events in slightly different words, and unless you are extraordinarily persistent you may never find a website that gets its information from a different source. See my posts on Google and Spin (1, 2).

---

@roygrubb raises a related concern - that Google remembers and is influenced by our previous searches. This is not just a privacy issue but also a context issue - our interests may switch from one project to the next. For example, let's say I'm working on a project using SAP, so when I'm searching for technical information on this project I may concentrate on material that is relevant to SAP. But I certainly don't want Google to put an implicit SAP filter on my searches, even if some Google engineer thought this would be helpful to me, because that could seriously prejudice my view of the available technology. Worse, this bias might persist (without my knowledge) when I'm working on a completely different assignment.

I can imagine that Google could build some kind of context-awareness into its search algorithms, so it somehow detects when I move to another project. And (to take a more controversial example) if I search for information about some deadly disease, it can try and work out whether I'm suffering from the disease myself (in which case it can sell me health insurance before it's too late) or enquiring on behalf of a friend or client, or whatever. But that's not the point. The point is the increasingly complicated relationship between our tools and our knowledge, which even many technologically literate people seem touchingly naive about.

What's in a name? Google

There are various ways of comparing Google with Yahoo. Here are a couple, from the comments to dot.life, the BBC technology blog.

• Yahoo, well I can't remember the last time anyone said "can you Yahoo this for me?", everyone and anyone uses Google. [twelveightyone]

• yahoo.com ... is still the most visited website on the web, as it has been for a long time. [iyatoni]

While Google undoubtedly has some advantages over Yahoo, the common use of "Google" as a verb is not a conclusive factor. People often "hoover" using rival vacuum-cleaners, and they "xerox" using rival photocopying machines. The word "YaHOO" doesn't make a good verb - the accent is in the wrong place. If you want your brandname to become a generic verb, choose a trochee - FEDex, GOOgle, HOOver, XERox.

I have little doubt that people will be "googling" long after Google itself has been supplanted by newer search engines. Or even by Yahoo2, if Ballmer's wildest dreams are fulfilled.

Feature Interaction at Google

Google owns Blogger and Feedburner. So when I added a Blogger-related Feedflare to a Feedburner Feed of a Blogger blog (why did I think this was a good idea? don't ask!), I hoped this would be pretty painless.

Unfortunately not.

Feedburner gave me an error:

The URL you entered does not appear to be a valid feed ...

And when I went to look at my feed to check the problem, I got an error from Google

We're sorry ... but your query looks similar to automated requests from a computer virus or spyware application. To protect our users, we can't process your request right now.

Sigh. It looks as if this combination of features is triggering some auto-immune response. Is my blog going to come out in spots, or be laid up for a few days? I hope not, but you never know.

Of course this kind of thing is depressingly common. We can never assume that two lumps of software or service will interoperate cleanly, just because they are owned and operated by the same software company. At least some software companies (including Google) do make an effort to integrate and rationalize after acquisition; some don't even try.

And then there were two ...

So Microsoft wants to buy Yahoo!, huh? The BBC describes this as a shotgun wedding - with Google wielding the shotgun. [BBC News, February 1st, 2008] And in a post entitled Monkey Boy's three-legged race, Fake Steve Jobs reminds Steve Ballmer of his previous disdain for mergers. When such mergers involved companies getting together to compete against Microsoft, maybe Microsoft could afford to be confidently superior. But now it seems it's Microsoft that needs a merger (with Yahoo!) to compete against the market leader (Google), and Ballmer's previous words may come back to haunt him.

(For another deal like this, think of Oracle buying PeopleSoft to compete with SAP.)

Google purports to be upset at the deal, with a pompous protest from its Chief Legal Officer entitled Yahoo! and the future of the Internet, and even the often cynical Fake Steve seems to take this protest at face value. But if Fake Steve's own analysis is correct, Google has no need to worry. Challenging the deal may simply be a way of making sure the Microsoft board can't back down without losing face.

Apart from Google, Microsoft and the Yahoo! shareholders, who are the winners and losers in this game? Bill Burnham thinks that this is a Bad Deal for Silicon Valley, because Yahoo! was one of the prime buyers of internet startups (notably del.icio.us and Flickr). But of course there are plenty other players. eBay is perhaps still licking its wounds after the over-priced acquisition of Skype, and NewsCorp (which has ruled out a rival bid for Yahoo!) maybe hasn't yet quite worked out what to do with MySpace, but that leaves TimeWarner (owner of AOL), Comcast (owner of thePlatform), IAC (owner of Ask and Bloglines) and a few others.

Oh yes, AOL-TimeWarner, that was a merger wasn't it? The $200bn company that was created by the take-over of media giant TimeWarner by the Internet upstart AOL, but the letters AOL no longer part of the company name, and AOL has now reverted to its ordained place in the corporate world, as a division of a large media company called TimeWarner. A number of the AOL local operations have been sold off, and Google currently has a 5% stake of the remainder [Press Release].

In the past, TimeWarner has got rid of many once-profitable divisions, including Atari, MTV Networks, and Time-Life. Some investors have demanded a break-up of Time and Warner, so perhaps a return to the good old days of Warner Brothers. That kind of thing seems to be normal ebb and flow among media companies. Companies can merge, but they don't necessarily stay merged.

But we haven't seen much of that in the software industry yet. To date, IBM and Microsoft have successfully defended themselves against regulatory break-up. (But the future demands of Wall Street may be more difficult to ignore.) The post-merger Microsoft will be a different kind of company, with new challenges. Maybe Steve Ballmer needs to take Larry Ellison out to lunch and pick his brains. (And I never thought I'd say that.)

Update (further commentary)

• If apparently intelligent people/organizations do apparently stupid things, it is tempting to look for some secret conspiracy or hidden motive that will make sense of the plan. For example, the Economist thinks this might be a devious trap to get Google snared in antitrust action. [The Economist (Feb 5th), via Fake Steve Jobs]

• Marc Andreessen agrees with me that there are plenty other companies acquiring Internet property, but Bill Burnham still thinks the overall effect is negative for Silicon Valley. Fred Wilson sums up: It's time to think long-term.

• Meanwhile Fake Steve Jobs is enjoying himself. Some months ago he was complaining of boredom, pining for a really good train-wreck merger. A reader asks if the Microsoft / Yahoo deal qualifies for this description. FSJ's answer is a resounding Yes.
  

Another Update

• Fake Steve has finally worked out the truth: Google is secretly praying for the Microhoo deal to go through

Questions, Questions

Eric Schmidt is quoted by the Financial Times (May 22, 2007):

"The goal is to enable Google users to be able to ask the question such as 'What shall I do tomorrow?' and ‘What job shall I take?’ "

With something as outrageous as this, A-List bloggers such as BurningBird and Nicholas Carr hardly need to say anything themselves. They simply post the quote under a suitably provocative headline and get their readers to do the detailed commentary. Nice.

But what Schmidt actually said is clearly false, because Google users have always asked these questions. The problem has always been getting half-decent answers. Possibly Schmidt's real goal is that Google shall attempt to provide answers, or (even better) get some other company to pay for the opportunity. Or that Google shall influence the questions we choose to ask. Or even that we choose to channel every single question in our lives in Google's direction.

But it is interesting that Schmidt expresses his goal in this way, because it illustrates the way Google blurs the boundaries between itself and its users. In his own blog (Google is Me, The Joy of Personalization), someone called Maluke suggests that such blurring is commonplace.

"It's not specific to Google actually, it’s an instance of a common kind of delusions of grandeur for big software companies, media outlets etc."

This relates to an earlier discussion on Google and God from November 2003, where I wrote:

"Many believers say God is not sitting on a cloud somewhere, God is in ourselves, in our hearts. When Google is equated with God, we are supposed to interpret this equation as referencing not the Google software nor the Google company, but the Internet community as a whole - ourselves as Google users. And perhaps we geeks are supposed to be flattered by this."

A few short years later and the public mood about Google has certainly shifted hasn't it?

Website Metrics

You get what you measure. And one way to influence behaviour is to provide an easy metric that is aligned to the behaviours you want to encourage.

This is as true in software as anywhere else, and is one of the reasons why software metrication is an important aspect of software quality.

In a post How to Misuse Google Analytics, Seth Godin points out that Google Analytics measures the success of a website from Google's perspective. Google's commercial objectives involve things like maximizing advertising revenue.

But as Seth points out, the quest for traffic can cause a website designer to make bad decisions. While many websites may benefit from advertising revenue, most non-spam websites have other objectives as well - for example, disseminating and championing new ideas. Google Analytics does not provide metrics relevant to these objectives.

Thanks to Google Analytics and related mechanisms, Google provides a positive feedback loop that reinforces its own commercial agenda. Systems thinker Donella Meadows identified the provision of such positive feedback loops as one of the ways of exerting power and influence over a complex system. (Wikipedia summary, original paper), and anyone who wishes to counteract this kind of power and influence should study her paper carefully.

The End of Google?

In November 2003, I saw the writing on the wall for Google.

"Google will therefore support, with ever-greater efficiency and effectiveness, an intellectual activity characterized by A.A. Milne (author of Winnie-The-Pooh) as 'Thinking with the Majority'." [Google and God. See also Scribe's post Drawing Parallels]

Some more recent doomsayers for Google:

• Jason Kolb: Google is Dying a Microsoft Death (via Willems van den Ende)
• Jason Kolb: More Evidence ...
  
• Igor Palmer: World Wide Web of Junk (via Kingsley Idehen)
  
• Igor Palmer: The Information Age is Seriously Ill

Jason argues that the quality of search results is being eroded by forces outside Google's control. Meanwhile, Igor thinks that "the information age – as aged and ill as it is now - isn't fading away fast enough".

But these voices are still in the minority. For the time being, Google is still the search engine of choice for the Internet Majority, for reasons explained by Sean McGrath: "The next best thing to knowing something, is knowing where to find it."

Sean avers that bookmarking is obsolete, because you can always use Google. He gives a few examples of his use of Google.

• "I wonder is there a direct flight from Knock to Birmingham?"
• "Where in the world is Myanmar?"
• "Have Australia ever qualified for the Soccer World Cup?"

I agree that Google is okay for enquiries of this kind - although I'd probably prefer Wikipedia for two of them. But I'd regard these enquiries as information rather than knowledge.

According to Sean: "In a short space of time search engines have literally changed the way we think about knowledge." It may well be true that many people have started to equate knowledge and its discovery with the results of a Google search. If so, this is a regrettable trend.

[RV posts on Google] [del.icio.us links on Google] [Technorati on Google]

Search Engines

For how much longer will everyone be happy using a Google-style interface (text box input, sorted list output)?

For a while now, I've been using Kartoo as my preferred search engine, because I rather like the way it visualizes the search results. (Thanks to JJ.)

Jef Newson recently pointed me towards Google Music Search. But this is just another text box. How do I enter the song I want to search for? Why can't I hum into a sound file, or pick out the tune on a keyboard?

Is this so far-fetched? Look at the Retrievr experimental service, which uses a rough sketch to search through images on Flickr.

And finally, look at the concept of Brain Fingerprinting, which apparently searches through a human brain to perform pattern matching against memories of an event or non-event. (See BBC News and ABC News via Guerrilla Innovation and Emergent Chaos.)

The battles between Google and Microsoft (and the rest) are fascinating, to be sure. But these platforms are not about to provide support for the things we are really searching for - original ideas, new customers and markets, reliable knowledge, and so on. Intelligence tools may be able to search for complex patterns (chemical compounds or social clusters possessing certain structural characteristics), and visualization tools can reflect these back, but it's usually pretty hard to define what you want or to interpret the results.

So we have to translate from what we really want into what we think we can get. Search for the title of a song, or a snatch of lyrics, or some other characteristics, and these search engines can probably find something relevant. Search for a topic, and these search engines will find companies that want to sell you their expertise. Search for what everyone else is thinking and saying and reading and buying, and it will come top in the search rankings. ("Thinking with the Majority"). What more could we possibly wish for?

Technorati Tags: Flickr Google Kartoo Retrievr search visualization

Focus

I have issued a number of probes to BI vendors for views on the potential synergies between SOA and BI - what I've been calling Service Oriented Business Intelligence (SOBI).

Service-Oriented Business Intelligence (SOAPbox blog)
Web Services to Improve Business Intelligence (CBDI Journal, June 2003)
Service-Oriented Business Intelligence (CBDI Journal, October 2005)

This week I had a useful briefing from Information Builders, makers of WebFocus. I first encountered Focus when I was working with Fourth Generation Languages (4GL) over twenty years ago, and it's interesting to see how the present stance of Focus (which I must now remember to call WebFocus) represents both continuity and change.

Information Builders certainly seem clued up about web services and SOA, and claim to have been playing in this space rather longer than some other BI vendors.

SOA

IB's main entry into the SOA space is a product called iWay, which calls itself an "Adaptive Framework for SOA" and claims to be "a complete toolset for creating composite applications and reusing existing IT assets". iWay is marketed by a separate company, iWay software, which is a wholly owned subsidiary of Information Builders.

One interesting piece of functionality is that it can be used to make data from legacy systems (such as unstructured data from notes fields) visible to the Google Enterprise Server, and therefore available for aggregation and analysis. May be worth a look.

Value of BI

4GLs were always supposed to improve productivity for developers, and to improve access for end-users. This twin agenda clearly remains in force. Information Builders sees business intelligence as a key source of value to any business organization; so the greater the number of people using the BI tools, the greater the potential value to the business. (Well, they would say that wouldn't they?) There is therefore considerable emphasis on improving the accessibility of BI functionality, as well as achieving economics of scale in the delivery of BI functionality.

Of course, technology vendors are naturally prone to make optimistic statements about the value of their tools, and the importance of having everyone using them. In this case, realistic assessments of the value of BI must depend on analysing the potential to improve business processes. Information needs are integrated with specific business responsibilities. Business processes and services may be improved by introducing effective feedback loops - for example if you allow customers to access restaurant inspection data, the dirty restaurants disappear pretty quickly. Note that these control loops typically go outside the boundaries of a single organization.

Instead of business intelligence being a highly specialized function, restricted to head office wonks with expensive and complicated gear, the power of business intelligence is taken to the edge of the organization - together with the corresponding accountability.

Integrated BI

I agree that the business value of BI may be greatly enhanced when BI is integrated with the business process. I have been calling this integrated BI; Information Builders talks about operational BI or pervasive BI, which are perhaps not quite the same thing, but are at least broadly in the same area.

There are lots of integration techniques that are relevant here - not just web services, but also Web 2.0 technologies such as Atom and RSS. For example, it is possible to subscribe to a complex enquiry. But we can push this further - imagine being able to subscribe to a hypothesis, and then being notified whenever any relevant evidence (for or against the hypothesis) becomes available.

Collaborative BI

I have argued that the next step beyond integrated BI is collaborative BI - supporting collaboration between distributed knowledge workers. Having implemented simple versions of Embedded BI and Integrated BI, Information Builders has announced the intention to introduce some support for Collaborative BI in the 2006 release of WebFocus. I look forward to seeing the details of this. However, I suspect that the full power of Collaborative BI will take longer to develop.

BI Process

So how is all this technological product innovation going to be reflected by process innovation - affecting the way that people build and use BI systems and services? Perhaps it is too early to say. Vendors like Information Builders may contribute to this innovation, may disseminate patterns and best practices, and may wish to develop formal methodologies. But I suspect the important changes will emanate from the user community, and will be slower to emerge.

Bloglines Upgrade

A fascinating insight this week into the processes of software innovation.

On October 10th, Redmonk industry analyst Stephen O'Grady announced that he was switching his newsreader away from Bloglines. His main gripe against Bloglines was a lack of visible innovation. Having rejected the Google alternative because

the interface seems to want to break down my preference towards browsing by feed to a river-of-news/relevance style interface

(in other words, it would force him to change the way he works - does this count as arrogance or innovation on Google's part?)

he selected an experimental newsreader (Feedlounge) from a couple of Redmonk friends. The single most compelling feature for him was the fact that he can navigate the entire UI via the keyboard.

On October 13th, Bloglines announced some new functionality. The first item on the list? You guessed it - hotkeys. You can now navigate your feeds using the keyboard.

Unless this is an amazing coincidence, this represents some impressive influence on Stephen's part. I'm very envious - I'm always telling vendors how they could improve their products, but I've never had any of my suggestions actually released within three days.

And it also represents an impressive degree of responsiveness within Bloglines. Okay, perhaps it didn't require a major product redesign, and perhaps there was already something in the pipeline, but a three day response is still pretty good.

Well done Stephen, well done Bloglines. Stephen may prefer to use the experimental stuff, but at least for the time being I'm going to continue using Bloglines.

Received Opinion

Opinion is recursive. I have always been wary of technical research based on opinion surveys, because most technical opinions are already based on second-hand information. How are we to interpret the fact that 75% of IT managers believe that platform X is flexible, or that vendor Y is attacking vendor Z, or that the most significant success factor is P? How many of these opinions are based on first-hand experience, and how much on reading what other people think – or what other people want them to think.

(The internet amplifies this effect. Google typically finds you the information that other people have already found; blogs and wikis often circulate received opinion without critical engagement; vested interests are often imperceptible behind a complex information supply chain).

There has recently been a vigorous debate between Lotus Domino and its admirers on the one hand, and the Radicati Group. John Vaughan argues that the Radicati Group analysis is based on out-of-date opinion; Lotus is now under new leadership, and the opinions collected by Radicati don't reflect this.

I haven't looked at the rights and wrongs of this particular debate, but I want to comment on some general principles of software industry analysis.

Clearly there is often a delay between a real change and the perceived change. Outsiders don't change their opinions just because there is a new management team with some fine-sounding words. And even when the new management has started to achieve something, it may take a while before this disseminates through received opinion.

Given that there is always a huge legacy of out-of-date opinion, the challenge with opinion surveys is to detect and interpret shifts in opinion, rather than simply report the current distribution of opinion.

Obviously the vendor marketing departments care about the prevailing opinion, because this is what they are selling into. Purchasers care about prevailing opinion for a different reason: because they don't want to look stupid buying a technology that nobody else wants. But in some cases, the ideal scenario for a purchaser is to buy just before everyone else buys. You may be able to negotiate deep discounts and lots of free support from a vendor who is hungry for revenue and for customer success stories. Then if everyone follows your lead, you may appear very smart indeed.

Just like in the stock market, good decisions are based on a shrewd estimate of tomorrow's prevailing opinion. Momentum only works in the short term. And if we analysts (whether stock analysts or technology analysts) simply reflect momentum, we aren't doing our job.

---

Related posts: Industry Analysis by Survey (July 2009), On Practices and Pitfalls (September 2009), On the Transparency of Algorithms (October 2016)

Google and God

Anyone who seriously equates Google and God must be both intellectually and spiritually impoverished.

originally posted November 1st, 2003

---

Is Google God? (Thomas Friedman, June 2003)
There, but for the grace of Google (Ben Macintyre, May 2005)
Stick to what you don't know (David Carr, May 2005)

Is Google omniscient? Only if all the knowledge in the world is accessible via the Internet, and Google provides reliable access to this knowledge. Neither of these conditions is valid.

Google and GooglePlex represent very large finite numbers. There is always a finite number of results from a Google search, and only a fraction of these are factual, meaningful, relevant or valuable. In contrast, God is usually conceived as infinite.

> David Kolb on fact, meaning, relevance and value

Many improvements to Google have been suggested, and some of these might improve the fact, meaning, relevance and/or value of a search-act. Google becomes a kind of Panopticon, watching the whole world watching the whole world.

> more on the the limitations of the Panopticon
> suggested improvements to Google (pdf)
(compiled by Seth Godin and Ramit Sethi)

The suggested improvements are just great for those people who want to ask the same questions as everyone else, and get the same answers. Google rankings already depend on the clicks of previous websurfers, and this dependency will become more sophisticated. Google will therefore support, with ever-greater efficiency and effectiveness, an intellectual activity characterized by A.A. Milne (author of Winnie-The-Pooh) as "Thinking with the Majority".

Many believers say God is not sitting on a cloud somewhere, God is in ourselves, in our hearts. When Google is equated with God, we are supposed to interpret this equation as referencing not the Google software nor the Google company, but the Internet community as a whole - ourselves as Google users. And perhaps we geeks are supposed to be flattered by this.

But however much of our minds and our lives we publish on the Internet, lots of stuff - perhaps even the most important stuff - cannot be published, cannot be put into TXT and JPEG.

And the things that are really worth searching for - integrity, wisdom, justice, courage and love - are not simply listed on Google but can only be found through focal practices, and through authentic engagement with other people.

> David Kolb on integrity, wisdom, justice, courage and love

> Albert Borgmann on focal practices

[RV posts on Google] [del.icio.us links on Google] [Technorati on Google]

Google and Longhorn

How does an act of branding reveal problems with both Google and Microsoft?

Originally posted October 31st, 2003

---

Popular commentary suggests that Microsoft's new generation operating system (code-named Longhorn) represents a threat to Google. There are lots of blogs discussing how Google should respond. But there is a deeper issue.

> suggested improvements to Google (pdf)

(compiled by Seth Godin and Ramit Sethi)

If you search for "LONGHORN" on Google you will get loads of references to Microsoft's latest software, at Microsoft and third party websites. You will also find some websites referring to Longhorn cattle and Longhorn sheep, and some of the places they can be found.

We must assume that Microsoft marketing people were aware of these associations, and of the opportunities they provided for some mild humour at Microsoft's expense. After all, it is standard practice for IT-literate marketing people nowadays to run a proposed brand name or project name through Google, to see what comes up.

But now try putting "LONGHORN BEETLE" into Google. You will now find a load of other websites, referring to various species of a highly destructive pest. I imagine that Microsoft marketing people were not aware of these associations.

Of course, if you had scrolled through pages and pages of Google search, you would have come to the Longhorn beetle eventually. But the point is that most people only look at the first few (or few dozen) pages. If you don't already know about Longhorn beetles, Google will not draw your attention to them.

Anyone who uses search engines regularly is should be aware of their limitations. While there are concerns in some quarters about Google's increasing dominance of the Internet search space, this isn't just about Google, since competing search engines may suffer from the same limitations. (While competition is important commercially, it doesn't always give us a genuine choice.)

How did I find out about longhorn beetles then? I looked in an old-fashioned reference book. Not very difficult - but the problem is that using Google (or Internet search engines generally) is so easy that people don't always remember to use other sources as well. Google was okay once I knew more precisely what I might be looking for.

This is one of the pitfalls of using the Internet for serious work. It is not a serious pitfall, as long as people are aware of its limitations, and take sensible precautions. But in its corporate enthusiasm for the Internet, Microsoft is often one of the first organizations to fall into any Internet pitfalls. 

 

Possible trends

• Google becoming dangerous - when people start to act as if the Internet was the only available source of information, and Google the only way of finding stuff 
  

• Microsoft remaining dangerous, Google becoming vulnerable - as long as Google represents a source of Internet value as yet untapped by Microsoft 
  

• Microsoft remaining vulnerable - when its employees themselves get carried away by the wonders of the Internet, and fail to take ordinary precautions. 

See also You Don't Have To Be Smart To Search Here (November 2008)

More posts about Google 

• https://demandingchange.blogspot.com/search/label/Google
• https://posiwid.blogspot.com/search/label/Google
  
• https://rvsoapbox.blogspot.com/search/label/Google
  
• https://rvsoftware.blogspot.com/search/label/Google

 

Updated 12 May 2021