Wednesday, November 05, 2014

Inside the Whale

The ethics of "pay-to-play" are being explored in the video games market. Some "free" games are funded by optional in-game purchases, such as extra weapons. There are several problems with this. Firstly, the availability of ever-more-powerful weapons might distort the game. Obviously a wealthy player could beat any chess grandmaster in the world, if the game of chess allowed you to buy an extra queen from time to time. In which case, what is the point of playing at all?

Secondly, a small minority of players are funding everyone else. In the video game world, these are known as whales. Video games often come out in multiple (overlapping) editions, and some whales can be persuaded to buy every edition. So the game developers and the majority of the players are exploiting the whales. That's fine if they can afford it, but some of the whales may be tempted to spend more than they can afford.

Some software vendors have accused industry analysts of operating a "pay-to-play" system, which allegedly favours those vendors who are willing to buy into the latest jargon (hype). And some analyst firms have a tendency to fragment any given technological marketplace into subdivisions in order to sell more "editions" of expensive reports, and to provide more differentiated opportunities for vendors to earn (or allegedly purchase) a good rating.

So if one were to produce a 2x2 matrix based on Commercial Ambition and Willingness to Invest, then the top right quadrant could be called Whales.

"If you're not familiar with how the analyst industry works, here's a very short summary. Companies brief analysts on their plans, the state of their businesses and the products they are bringing to market. Some companies pay the analyst firms to have a two-way conversation and get advice from the analysts. Then the analysts write up information-rich reports about some important business trend or another, based on all the briefings they've done. They sell those reports for hundreds or thousands of dollars to consumers wanting to benefit from all the research performed by the analysts." (Source ReadWrite Web August 2009)

Is there a pay to play problem in the analyst industry?, You told us which analyst firms were most, and least, independent (Influencer Relations, February 2014)

Tuesday, May 13, 2014

What shape is the internet (continued)?

@ironick and I have been arguing about the shape of the internet since my September 2010 post on this subject. Over the past few days, we have returned to this topic on Twitter. Nick has captured the latest tweets in his Storify piece The Shape of the Web - Database Wars Redux.

The argument was triggered by @djbressler's observation that some new browsers (including an experimental build of Chrome) were hiding the URL from the user. This is a reflection of the fact that users increasingly type "Amazon" into the browser rather than "" let alone "". Presumably, hiding the URL will further encourage this trend.

Google and other search engines appear to benefit from this in two ways. Firstly, it increases the already heavy dependence of the ordinary internet user on the search engine. And secondly, every time an internet user navigates via search rather than via URL or hyperlink, the search engine gets another opportunity to present some advertising, as well as collecting more information about that user.

Obviously, Google itself depends on URLs and hyperlinks. As Nick points out, Google still relies on links to construct its index, and still uses a version of the original PageRank algorithm to influence what you see when you search for a given term. But indexing and search ranking are only loosely coupled to one another.

And nowadays, the search order is not solely determined by PageRank. Instead, the search order is increasingly influenced by browsing behaviour - of others as well as our own. If you ignore the first two items, click briefly on the third item, and then immediately return to Google to look at the fourth item, Google may conclude that the first three items weren't very relevant to you. In other words, this counts as a "vote" against those items.

Meanwhile, Google only had exclusive rights to the original PageRank patent (which belongs to Stanford University) until 2011.

Obviously Google is not completely open about these algorithms, because it is perpetually at war with SEO and spammers who want to get some commercial advantage by "gaming" the system. So there is a degree of speculation involved in working out what exactly Google is up to. Sometimes Google merely seems to appeal to the lowest common denominator, as David Auerbach suggests in his review of Metafilter search results ("Deranked"). However, it is beyond speculation that Google's behaviour has become increasingly sophisticated over the past decade, and that what we see is increasingly "personalized".

Nick accuses me of "confusing the use of behavior IN the ranking algorithm itself with using behavior to verify the quality of the algorithm". However, there is some evidence that Google initially trials new factors in parallel with the existing algorithm, before integrating these factors into the algorithm itself. (See for example, Google Panda.) In any case, the total behaviour of Google can be thought of in terms of the collective intelligence of human brains AND algorithmic software, and it may not be possible for outsider observers to be exactly sure where the boundary lies at any point in time. (We can detect "momentum", but not "position".)

Obviously URLs are not going to disappear entirely. For my part, I have always made an effort to use links and bookmarks rather than pander to the commercial interests and cognitive distortion of search engines. I don't think this undermines my general point - that the Internet-in-use (based on majority habits) is taking on a different shape. Obviously it is still possible to use the Internet in a disciplined and self-conscious manner, which Nick (always) and I (sometimes) practise, but the fact that this requires effort and intelligence makes it likely that it will never become mainstream.

In the long-term, Google may face a paradox. If people stop using URLs, then Google's ability to index and rank pages across the internet might possibly be compromised. But I'm sure that the clever people at Google have thought of this paradox, and already have a cunning plan.

Meanwhile, the internet (as experienced by ordinary users) is gradually becoming less web-shaped and more star-shaped, with your favourite search engine or social network at the centre. (Please note the word "gradually".)


David Auerbach, Deranked - Why has Google forsaken MetaFilter? (Slate May 2014)
Bill Slawski, The New PageRank, Same as the Old PageRank? (March 2012)
Daniel Sour, It Knows (LRB October 2011)

Related posts

What shape is the internet (September 2010)
What shape is your intranet (May 2014)

Updated  17 May 2014

Bring Your Own Expectations

Once upon a time, there was a clear separation between Work and Home. This separation has been undermined by two phenomena.

1. Working at Home - in other words, allowing work to invade the home environment
2. Bring Your Own Device (BYOD) - in other words, allowing personal devices to invade the work environment

In this post, I want to talk about a third phenomenon, perhaps more invisible but no less important. Bring Your Own Expectations means that we have all become accustomed to getting what we want from the Internet, and therefore expect to get the same things (or "affordances") from corporate systems and platforms.

One of the most obvious gaps between our expectations and corporate reality is the failure of search. The Internet has an uncanny knack of guessing what we want, and there are strong commercial incentives for Amazon, Facebook, Google and the rest to improve their "mind-reading" capabilities.

In comparison, your company intranet is simply not in the same league, and therefore cannot anticipate your needs in the same way. Some people see this as merely a technical lack, to be addressed by some functionality inside the company firewall that roughly resembles the way Google worked ten years ago. But this is far more than a mere technical shortcoming.

And search is just one difference. There are also expectations about interoperability. For example, do we expect to use one network for linking with colleagues and customers, and a different network for linking with friends and family? Most people are still learning how to manage these different worlds without getting muddled - for example, people who automatically put kisses onto private messages may find themselves carrying this habit into corporate communications. Maybe sometimes our expectations lead us astray.

Some service providers (including notoriously Facebook) insist that you have a single identity for personal and business use. Other service providers accept that people may wish to have two or more accounts, in order to keep personal and business use separate, and are happy to design premium services largely for the business user. A good example of this is DropBox for Business, which allows multiple accounts (e.g. a business account and a personal account) to be synchronized to the same computer. However, people will still expect to have at least as much affordance in the business sphere as in the personal sphere, and will be unhappy if their employer provides (for example) corporate file-sharing services that are not as good as (say) DropBox. (Other file sharing services are available.)

Related Posts

BYOD Bring Your Own Device (Feb 2012)
On Working At Home (March 2014)
What Shape is Your Intranet (May 2014)

Updated 5 November 2014

Saturday, May 10, 2014

What shape is your Intranet?

@djbressler tells us he is work­ing on a thought-piece about the bifur­ca­tion of the intranets from the Inter­net. In the meantime, in a post called Burying the URL (May 2014), he comments on an experimental build of Chrome, which continues the trend of hiding the URL and encouraging people to use search instead. Obviously it benefits Google when people rely exclusively on search. But it's not just Google's Chrome that is doing this; Firefox and Mobile Safari are also going in this direction.

In my post What shape is the internet (September 2010), I said that shifting the emphasis from URL hotlinks to search undermines the idea of the internet's being web-shaped. This point is also made in a post by @apike, referenced by David and also called Burying the URL (April 2014).

URLs are the essence. They make hypertext hyper. The term “web” is no accident – it refers to this explicitly.
See also an excellent Twitter debate following @apike's tweet.

When David talks about bifurcation, he means that "enter­prise IT is diverg­ing enter­prise tech­nol­ogy from con­sumer tech­nol­ogy in a way that’s cre­at­ing two irrec­on­cil­able branches of tech­nol­ogy". He observes that most company intranets have a pretty lousy search facility.

But most company intranets have pretty lousy cross-linking as well. They are mostly just pdf graveyards stuffed with documents of indeterminate pedigree, which people are often reluctant to waste time searching (even if the search facility were better) because they don't expect to find anything of value.

Actually, you can't always find what you are looking for on the Internet either, and that has a lot to do with the limitations of search, but there are enough amusing distractions to conceal this fact. Surely we don't want our company intranets to copy the internet too closely?

And remember that the data revealing Enron's problems were cheerfully displayed on the Enron website. But nobody important had bothered to look at these documents properly. (Actually, a bunch of students had analysed them years previously and concluded that Enron was bankrupt. They probably got low marks for that assignment!)

There is an increasingly common belief that the tech­nol­ogy used inside com­pa­nies should work the same way as outside, should provide the same "affordance". This is not Bring Your Own Device but Bring Your Own Paradigm Expectations. I guess I should work on a thought piece about this.

Related posts

What shape is the internet (September 2010)
Bring Your Own Expectations (May 2014)

See also Steven Poole, The pdf graveyards can only expect an increase in their undead populations (Guardian 9 May 2014)

Saturday, April 26, 2014

Does Big Data Release Information Energy?

@michael_saylor of #Microstrategy says that the Information Revolution is about harnessing "information energy" (The Mobile Wave, p 221). He describes information as a kind of fuel that generates "decision motion", driving people - and machines - to make a decision and take a course of action.

We already know that putting twice as much fuel into a vehicle doesn't make it twice as fast or twice as reliable. (Indeed, aeroplanes sometimes dump fuel to enable a safer landing.) But Saylor explains that information energy is not the same as physical energy.

1. Information energy doesn't follow conservation laws. Information can be created, consumed repeatedly, but never depleted or destroyed. (Unless it is lost or forgotten.)

2. Whereas physical energy is additive, the energy content of information is exponential.

3. The value of information depends on its use, and who is using it.

Let's look at his example.

"Total wheat production for a single year is valuable information; but total wheat production for ten years, combined ten years of rainfall data and ten years of fertilizer represents thirty times more data droplets, but probably contains one hundred times more information energy, because it shows trends and correlations that will drive a greater number of decisions." (pp 221-2).

In other words, thirty times as much data produces a hundred times more information. He doesn't say this extra information MAY drive more decisions, he says it WILL drive more decisions. In other words, the Information Revolution (and our increasing reliance on tools such as Microstrategy's products) is a historical inevitability.

But is it really true that more data produces more information in this exponential way? In practice, there is a depreciation effect for historical or remote data, because an accumulation of small changes in working practices and technologies can make direct comparison misleading or impossible. So even if the farmer had twenty years' worth of data, or shared data from thousands of other farmers, it would not necessarily help her to make better decisions. Five years' data might be almost as good as ten years'.

Data is moving faster than ever before; we're also storing and processing more and more of it. But that doesn't mean we're just hoarding data, says Duncan Ross, director of data sciences at Teradata, "The pace of change of markets generally is so rapid that it doesn't make sense to retain information for more than a few years." (Charles Arthur, Tech giants may be huge, but nothing matches big data, Guardian 23 August 2013)

According to Saylor, the key to releasing information energy is mobile technology.

"The shocking thing about information is not how much there is, but how inaccessible it is despite the immense value it represents. ... Mobile computing puts information energy in hands of individuals during all waking hours and everywhere they are." (p 224)

What kind of decisions does Saylor imagine the farmer needs to make while sitting on a tractor or milking the cows? Obvious it would be useful to get an early warning of some emerging problem - for example an outbreak of disease further down the valley, or possible contamination of a batch of feed or fertilizer at the factory. But complex information needs interpretation, and most decisions require serious reflection, not instant reaction.

So it is not clear that providing instant access to large quantities of information is going to improve the quality of decision-making. And giving people twice as much information often leads to further procrastination. Surely the challenge for Microstrategy is to help people deal with information overload, not just add to it?

Furthermore, as I said in my post Tablets and Hyperactivity (Feb 2013), being "always on" means that you never have long enough to think through something difficult before you are interrupted by another event. There is always another email to attend to, there is always something happening on Twitter or Facebook, and mobile devices encourage and reinforce this kind of hyperactivity.

Saylor concludes that "the acid of technology etches away the unnecessary" (p 237). If only this were true.

Related posts

Service-Oriented Business Intelligence (September 2005)
On The True Nature of Knowledge (April 2014)

Updated 19 June 2014

Thursday, April 24, 2014

Predictive Analytics for the Smart Consumer?

#CW500 If merchants can use predictive analytics to get more out of the customer, why can't the customer use predictive analytics to get more out of the merchant?

In December 2012, I reported on a subscription-based service from, which predicted future retail price changes (based on retailers' past behaviour) and encouraged its members to use these predictions to optimize the timing of key purchases.

Many retailers have fairly regular patterns of seasonal price changes and promotions, which are designed to maximize the lifetime profitability of a product. This is particularly important for fashion goods and high-tech, which tend to have a high initial price and a low clearance price. However, if customers (with the help of advisory services such as start to game these price changes, then profit optimization becomes a lot harder to calculate. So this kind of advisory service represents a significant threat to retail profitability.

In September 2013, was acquired by eBay and effectively closed down. “This is an exciting opportunity to bring Decide’s expertise in data and predictive analytics to the worldwide commerce leader and empower over 25 million eBay sellers,” said Mike Fridgen, CEO of “We believe teaming up with eBay allows us to realize our mission of leveling the playing field in commerce.” (eBay 6 September 2013, Geekwire 6 September 2013)

In other words, taking the advantage away from the customers and giving it back to the sellers.

However, other customer-side predictive services may be still available, including (for Gasoline) and Kayak (for air travel).

Predictive Showrooming Dec 2012

Tricia Duryee, Says It Will Accurately Predict Prices or Your Money Back (All Things D, 19 April 2012)

Thorin Klosowski, Kayak Adds Price Forecasting to Predict Price Drops and Increases (Lifehacker, 15 Jan 2013)

Thursday, February 06, 2014

What's the Most Important Task for Microsoft's New CEO?

"Bill Gates’s first day at work in the newly created role of technology adviser got off to a rocky start yesterday as the Microsoft founder struggled for hours to install the Windows 8.1 upgrade" (New Yorker, 5 February 2014).

Lots of people have expressed amusement or sympathy about this. But the twist comes later in the story.

"After failing to install the upgrade by lunchtime, Mr. Gates summoned the new Microsoft C.E.O. Satya Nadella, who attempted to help him with the installation, but with no success."

What is really going on here? Has Nadella nothing better to do? One wonders whether Gates really didn't know how to install the software or whom to call, or whether he was just making a point about who was really the boss.

When someone as clever as Gates asks a stupid question, it may not be a stupid question at all. See my post On the Difference between Judges and Geeks.

A Microsoft spokesman said that Mr. Gates’s first day in his new job had been “a learning experience.” Yes, but for whom?

As they say in Italy, se non è vero, è molto ben trovato.

Wednesday, October 09, 2013

Does this chart prove the tech industry's lack of diversity?

Twitter's forthcoming IPO has drawn critical attention to its male-dominated board. Vivek Wadhwa, a fellow at Stanford’s Rock Center for Corporate Governance, recently complained about the lack of women in Twitter’s top ranks. Twitter CEO Dick Costolo responded in childish fashion, tweeting that “Vivek Wadhwa is the Carrot Top of academic sources” and that Wadhwa had a “propensity for silly hyperbole”.

Source: Twitter CEO hits back at gender bias rap (New York Post, 7 October 2013)

Clearly, Twitter is not an isolated case: many commentators have pointed out that this is a more general problem. For example, ValleyWag prints a chart of the corporate boards of the top 17 tech companies, created by Gawker's @JimCookeIII and @NitashaTiku, The Boards Are All White: Charting Diversity Among Tech Directors via @HuffPostTech

But hold on. These are not the top 17 tech companies in the world. They are not even the top 17 tech companies in the USA. Notable omissions from Gawker's list are HP and IBM (which happen to have female CEOs), and Intel (which has a female president). Here is Wikipedia's list of the largest information technology companies (retrieved 9 October 2013).

Obviously a few women in high places doesn't make the problem of gender imbalance disappear. But it really doesn't help the cause of diversity to airbrush out those women who have made it to the top of the tech ladder.