Does Big Data Release Information Energy?

@michael_saylor of #MicroStrategy says that the Information Revolution is about harnessing "information energy" (The Mobile Wave, p 221). He describes information as a kind of fuel that generates "decision motion", driving people - and machines - to make a decision and take a course of action.

We already know that putting twice as much fuel into a vehicle doesn't make it twice as fast or twice as reliable. (Indeed, aeroplanes sometimes dump fuel to enable a safer landing.) But Saylor explains that information energy is not the same as physical energy.

1. Information energy doesn't follow conservation laws. Information can be created, consumed repeatedly, but never depleted or destroyed. (Unless it is lost or forgotten.)

2. Whereas physical energy is additive, the energy content of information is exponential.

3. The value of information depends on its use, and who is using it.


Let's look at his example.

"Total wheat production for a single year is valuable information; but total wheat production for ten years, combined ten years of rainfall data and ten years of fertilizer represents thirty times more data droplets, but probably contains one hundred times more information energy, because it shows trends and correlations that will drive a greater number of decisions." (pp 221-2).

In other words, thirty times as much data produces a hundred times more information. He doesn't say this extra information MAY drive more decisions, he says it WILL drive more decisions. In other words, the Information Revolution (and our increasing reliance on tools such as MicroStrategy's products) is a historical inevitability.

But is it really true that more data produces more information in this exponential way? In practice, there is a depreciation effect for historical or remote data, because an accumulation of small changes in working practices and technologies can make direct comparison misleading or impossible. So even if the farmer had twenty years' worth of data, or shared data from thousands of other farmers, it would not necessarily help her to make better decisions. Five years' data might be almost as good as ten years'.

Data is moving faster than ever before; we're also storing and processing more and more of it. But that doesn't mean we're just hoarding data, says Duncan Ross, director of data sciences at Teradata, "The pace of change of markets generally is so rapid that it doesn't make sense to retain information for more than a few years." (Charles Arthur, Tech giants may be huge, but nothing matches big data, Guardian 23 August 2013)

According to Saylor, the key to releasing information energy is mobile technology.

"The shocking thing about information is not how much there is, but how inaccessible it is despite the immense value it represents. ... Mobile computing puts information energy in hands of individuals during all waking hours and everywhere they are." (p 224)

What kind of decisions does Saylor imagine the farmer needs to make while sitting on a tractor or milking the cows? Obvious it would be useful to get an early warning of some emerging problem - for example an outbreak of disease further down the valley, or possible contamination of a batch of feed or fertilizer at the factory. But complex information needs interpretation, and most decisions require serious reflection, not instant reaction.

So it is not clear that providing instant access to large quantities of information is going to improve the quality of decision-making. And giving people twice as much information often leads to further procrastination. Surely the challenge for MicroStrategy is to help people deal with information overload, not just add to it?

Furthermore, as I said in my post Tablets and Hyperactivity (Feb 2013), being "always on" means that you never have long enough to think through something difficult before you are interrupted by another event. There is always another email to attend to, there is always something happening on Twitter or Facebook, and mobile devices encourage and reinforce this kind of hyperactivity.

Saylor concludes that "the acid of technology etches away the unnecessary" (p 237). If only this were true.

---

Related posts

Service-Oriented Business Intelligence (September 2005)
On The True Nature of Knowledge (April 2014)


Updated 19 June 2014

Tablets and Hyperactivity

@CharlesBrett via @rwang0 explores the ways in which mobility is changing what and how we do it (Feb 2013). He talks about the greater convenience of the tablet over the laptop

"the tablet possesses a flexibility to ‘turn on and to turn off’ that was never true of the misnamed laptop"

and claims that this flexibility provides a counter-argument to the fear that they will invade and consume personal time

"In fact the reverse seems more likely.  You can be watching a movie and move to reading an urgent email, doing the research to reply to it and then return to your movie — all from where you are. "

For Charles, the tablet is an almost universal device.

"Tablets that connect enable you to do what you want, whenever you want.  That can be any or all of email, personal browsing, corporate browsing, information access, decision taking, reading, entertainment, etc.  Indeed, one of the attractions is that you can switch at will between any or all of these.  About the only activity you cannot do is document creation."

Actually there are some vital activities that you cannot do on either the laptop or the tablet - to think and reflect and understand. Being "always on" means that you never have long enough to think through something difficult before you are interrupted by another event. There is always another email to attend to, there is always something happening on Twitter or Facebook, and mobile devices encourage and reinforce this kind of hyperactivity. Some might call it addiction.

Does Cameron's Dashboard App Improve the OrgIntelligence of Government?

In November 2012, it was announced that a mobile app to aid in decision-making and day-to-day government affairs was being trialled by the prime minister.

UK Trialling App to Aid Government Decisions (BBC News 8 November 2012)

Here are some quick comments from Twitter

@lesteph PM's dashboard is at best pointless, at worst dangerous, unless his briefing system has fundamentally collapsed

@dominiccampbell he may as well have it, but pretending it's anything other than a partial view and mostly for PR is daft

@willperrin rather an antediluvian counsel of despair there then. back to 'ringbinders full of..' briefing

@6loss The "dashboard vs intelligence" debate? IMHO dashboards are useless without fast feedback on action.

In a subsequent discussion on Linked-In, @6loss and I discussed some of the intriguing questions raised by this news story.

Firstly, we were missing the imperative for real-time action and feedback. Obviously the Prime Minister needs to know whether job vacancies are going up or down, but the idea of real-time update is just ridiculous. Suppose that seventeen new job vacancies have been posted in Smartchester in the past twenty minutes, Are we supposed to believe that these seventeen vacancies urgently need to be communicated to the PM so that he can take appropriate action?

What does make sense is a dashboard that supports an OODA loop. A well-designed dashboard should not only provide aggregated data, but also provide some way of making sense of the data. (It is possible that the data visualization may help here.) And then taking rapid action.

But in a well-designed organization, the responsibility for rapid action is delegated to the people in the front line, who are given the real-time intelligence and the resources/tools and the authority to solve problems effectively and efficiently. This is what the military call "Power to the Edge". A completely different order of intelligence is required at the centre, usually operating at a much slower tempo.

And since managers are often tempted to meddle with randomly varying processes (Deming called this "tampering"), a well-designed control system deliberately hides much of the volatility from senior management. (In cybernetics, this is called "attenuation".)

Secondly, I'm wondering what kind of statistics we are talking about here. When people talk about "statistics", they often mean the kind of statistics kids learn in primary school (totals and averages) rather than the kind of statistics kids learn in high school (correlation and significance). I wonder how many ministers could cope with high school statistics (let alone degree level) without a civil servant or adviser there to explain it to them? The danger of the "dashboard" is that it may eliminate the vital step of interpretation and sense-making, which is surely essential to evidence-based management. 

Thirdly, I'm wondering about the planned rollout of this App. Are we to suppose that all ministers and senior civil servants are going to be watching the same set of indicators, or does collective responsibility entail that each minister is watching a different set of indicators? In a typical control room, there are many people each watching a different dashboard or controlling a different sector: it would seem a bit redundant if they were all watching the same one. Meanwhile, the supervisor sits in his cubicle playing Angry Birds, or sending texts to his neighbours.

A few weeks after this discussion, writing in the New York Times, Will Wiles compared this dashboard with the Viable Systems Model implementation in Chile under Salvador Allende. He pointed out that the dashboard is not truly cybernetic because it lacks a mechanism to translate all that data into action. Quite so.

Will Wiles, Before Fruit Ninja, Cybernetics (New York Times, 30 November 2012)

 

Update: See also Shannon Mattern, Mission Control: A History of the Urban Dashboard (Places Journal, March 2015)

Predictive Showrooming

Retailers are being urged to adopt big data and predictive analytics as a defensive weapon against showrooming - the growing phenomenon of customers looking at a product in a store and then ordering it online.

For example, TIBCO suggests "companies that are most effective at combating this type of new threat are those who have used data analysis and predictive analytics to predict what type of engagement will resonate most effectively with their existing and new customers". Meanwhile, @ericylai talks about marrying Big Data and Mobile, while @larryfreed talks about providing a unified, cross-channel experience.

However, @barneyjopson reports a new twist. Consumers can now use big data and predictive analytics themselves, by using a service from decide.com, which predicts future retail price changes (based on retailers' past behaviour) and encourages its members to use these predictions to optimize the timing of key purchases.

Update: In September 2013, decide.com was acquired by eBay and effectively closed down.
See Predictive Analytics for the Smart Consumer? (April 2014)

---

Larry Freed, 5 Tips to Turn Showrooming Consumers into In-store Customers (Dec 2012)

Barney Jopson, Torn between loyalty and a bargain (Financial Times Dec 2012)

Eric Lai, How Brick-And-Mortar Retailers Can Beat 'Showrooming' And Amazon.com (ZDNet Oct 2012)

Bryson White, Did Black Friday showrooming overshadow Cyber Monday shopping online? (Adobe Digital Marketing Blog, Nov 2012)

Data Analysis Versus ‘Showrooming’ (TIBCO Spotfire Blog Nov 2012)

See also Showrooming and Multi-Sided Markets (Dec 2012)



Updated 23 April 2014

Email newsletters

"Is email dead?" asks @Graham_Walsh (via @nigelwalsh @leebryant ) noting that Ben & Jerry's are stopping their newsletters, and now using updates on Facebook & Twitter.

Email has always been a poor medium for broadcasting news, for several reasons.

Email is used for delivering many different kinds of content, ranging from important to rubbish. For most people, newsletters (even from companies we are interested in) are considerable less important than direct messages from customers, colleagues and friends, some of which may require an immediate response. Therefore even the most interesting newsletters are likely put aside for later reading.

Many of us receive hundreds of legitimate email newsletters - from an assortment of companies and societies and other organizations we have some vague association with - as well as loads of more spammy stuff. I have set up an automatic filter for the regular ones, which go into a folder called Newsletters, where they may sit for weeks or months before I find time to look at them. Many get deleted unread.

Traditional newsletters contain several items, but this is problematic on email. Most people won't scroll down beyond the first page without good reason. (This was a problem for a company I once worked for, which used to sell advertising space at the top of its newsletters to other commercial organizations, with the result that some readers only ever saw the advertisement and not our own content.)

There is also a design problem. Email clients are generally less sophisticated than browsers, and newsletters that look fine on one computer may be almost unreadable on another. As for reading a traditional newsletter on a mobile phone or handheld device - forget it. (Maybe the iPad?) So people end up producing text versions and HTML versions, and it just gets more complicated without actually solving the problem.

Finally, there is a problem with organizational innovation. Many people who claim to be leading-edge technologists seem mysteriously attached to email as a general-purpose communication mechanism, and reluctant to use the wide variety of alternative mechanisms that might serve a particular purpose more efficiently and effectively - not just when communicating with customers (who might be slow to adopt newer alternatives) but also with their own peers (who have no such excuse). Email becomes a regressive standard for all forms of communication, and there is little willingness to gather evidence about its effectiveness.

There is an expectation that corporate IT will drive innovation in matters technological. But in many large organizations I've worked in, it is the marketing department that is is more likely to drive this kind of initiative. The use of Facebook or Twitter may be a tactical initiative, adopted as an experiment and abandoned if the results are disappointing; but what I'd see as strategic for consumer-oriented marketing is having a flexible communication platform with strong feedback loops to support detailed customer analytics, and that's what I'd expect corporate IT to provide.

Or you could just stick to email, on the grounds that this is safe and familiar to all - business IT alignment interpreted as keeping IT inside the comfort zone of the business.

The Technology is not the Product

A perennial source of confusion in technology analysis is the false equivalence between technology and technical products. I refer to this as the first asymmetry. (See for example my article with Philip Boxer on Metropolis and SOA Governance.)

This confusion sometimes crops up in adoption studies, where people talk as if the adoption of a technology was the same thing as the adoption of a product, which is clearly not the case. Leading software companies may sometimes employ people to evangelize a generic technology (SOA, Complex Event Processing or whatever) instead of focusing specifically on their own company's own product range - this tactic allows evangelists from rival companies to collaborate in order to expand the credibility (and therefore the market) for the technology as a whole.

We should also note that complex products often contain a bundle of different technical inventions. As I pointed out in my blogpost on the Red Queen Effect, a product may be composed from a large number of components, each of which may be subject to technical innovation. Product innovation is not a simple linear function of technology innovation; a product lifecycle can be extremely short, but most of the underlying technology may be moving much more slowly. Or vice versa.

Thus a new device such as the iPad shouldn't be regarded as a single instance of "new technology"  but as a designed product that contains a large number of new and not-so-new technologies. If I buy a device that happens to contain special features for disabled users, based on the latest technology, can I be said to have "adopted" this technology even if I never actually use those features.

Meanwhile, some products fail to realise the potential of the available technology. For example, Plumen, a niche manufacturer of low energy lightbulbs, makes this claim about its competitors, saying "there are CFLs out there that provide a nice colour of light and that turn on quickly, but they are hidden under a sea of cheap, poor quality bulbs that are given away for free" [Plumen website].

Some types of regulation focus on products rather than technologies. See for example Australasian Biotechnology (Notes from November 1998).

---

Quotes

"The end product is not the technology but the experiences that we make possible." Griffiths on mobile, Brand Republic, March 2005. “One of our philosophies at MobiTV is the experience as the product, not the technology.” Paul Scanlan, interviewed by Connected Planet, November 2008.

"Whatever the product is, it includes the usability of the technology." John Unger Zussman, Docufictionalizing user manuals. InfoWorld, 15 Feb 1982. Available at Google Books. 

See also Sameera Banduk, Marketing Manager of Well.ca, explaining why his company received awards in every category except technology.

"We’re not seen as a technical company, because the technology is not the product. The technology is in the background, and it’s mind-blowing, but what you see from the website is the customer experience and the products." [Customer Focus and Geek Cred - An Interview with Well.ca. January 22, 2010]

And finally, see Interview with Sherry Turkle, Frontline, Sept. 22, 2009

Software Retronyms

A curious set of posts from my friends at Gartner on how technology complicates language

• anemic words such as wireless (David McCoy)
• anemic or retrospective, complementary retronym (Nick Gall)

When something new comes along, we sometimes have to invent new terms for the old things. Before we had mobile phones, all we had was phones. But once mobile phones became common, we needed to have a way of referring to the old phones, so the terms "fixed phone" or "landline" started to appear.

Once upon a time, all personal computers went under an office desk. Then we started to get portable computers, supposedly cool enough to put on your lap (if you happen to be wearing heat-proof trousers), so they are called laptops. The old-fashioned sort that sit under a desk are now called desktops.

Words like this, that are introduced for the sake of some kind of backwards compatibility, are called retronyms.

As an industry analyst, I often hear vendors trying to distance themselves from their competitors, or justify the wonders of their latest device by contrasting it with some notional predecessor. So they have to find labels to describe and disparage the past. Old software somehow manages to be simultaneously monolithic and spaghetti; old software methods are always silo-based waterfalls, and so on. If you are a serious innovator, the worst insult you can ever throw at anyone is "traditional".

Do we need terms for systems that are not real-time, not service-oriented, not event-driven, not business-aligned? Yes of course we do. But we may not find many vendors who will admit that their products lack any of these characteristics. On the contrary, they are all pimping up their products with the latest fashionable buzz-words.

Perhaps the most extraordinary example of retronymy is the series of mobile phone generations. First we had 3G. Then we had technology that wasn't quite 3G, so it was called 2.5G. Now we have 2.75G. Where will it end?

i-Phone or wii-Phone?

There are apparently two reasons someone might want a mobile phone, which can be summed up by the contrast between the Apple prefix (i-) (pronounced I) and the Nintendo prefix (wii-) (pronounced we).

The Apple prefix suggests private consumption. The iPhone appears to be an elegant cross between a top-of-the-range iPod and a Blackberry, designed for people that want to look cool while cutting themselves off from normal social interaction. Ever since the launch of the Sony Walkman, those tiny headphones signal "don't talk to me, I'm listening to something". And many people seem to use their mobile devices as a way of disengaging from their immediate surroundings.

The iPhone has been extensively reviewed, and I don't want to do a detailed review here. I just want to point to a few comments that suggest the iPhone isn't radical enough:

• "Call me crazy, but I think Apple have overdone the technology innovation, and undercooked the business model innovation." (Martin Geddes)

• "What it doesn't do is actually re-invent the very thing that makes cellphones magical: how you connect with other people." (Seth Godin)

• "But something is missing, something important drowned in this flood of f(x)." (Rikard Linde via Seth Godin on Web4)

The wii-prefix, on the other hand, suggests a shared experience. In a post Disappearing Telephony from January 2006, Martin Geddes made an excellent point about conversation and presence ("humans are sophisticated social animals, and it shouldn’t come as a surprise if our conversation tools need to act intelligently too"), which I followed up in my post on Coffee Shop ("Forget LinkedIn, let's have EspressedIn"). It now seems Seth Godin and Rikard Linde are thinking along similar lines.

As far as I know, Nintendo has no plans to launch a mobile phone. But there are some good precedents for social interaction in the latest games consoles, and it would be interesting to see a communication device based on the wii- prefix rather than the i-prefix.

Update January 2007

I have found some rumour pages from last year about a possible wii-phone, plus a German cartoon.

• Wii-mote to be Wii-mic and Wii-phone? (May 2006, Seth Bland)
• Wii-phone blog (Sept 2006)
• Cartoon (January 2007) via Andreas Schaefer

Update June 2019

Many years ago, I wrote a critique for New Society about the way enterpreneurs like Sir Clive Sinclair were being idolized by the Thatcher Government. As an aside, I noted the attenuation of social consumption.

"The trend is towards individual consumption, and all of Sir Clive's inventions contribute to this trend. He has produced a car that takes only one passenger, a television too small for communal viewing, computers that are mainly used by children playing private games on their own, thus avoiding the stress of direct competition with other children. Many people working with computers have spoken with enthusiasm of a future society in which all basic transactions (including schooling, shopping, banking and all office work) will be carried out from home, using computer technology. We have already substituted the personal hifi for the concert , the rented video for the evening out, the televised press conference for the public meeting. Now the trend is to reduce social contact even further." Cry God for Maggie, England and St Clive (New Society, 9 May 1985)

Obviously some of this argument looks a bit dated now. What about multi-player games, as I noted in 2007? What about "social media"? But I think the main thrust of the argument still applies. Let me give the final word to @rachelcoldicutt

I mean, the tldr for “why are we so individualised now?” is that it’s probably not because phones or civic design or spirituality or libraries or compassion fatigue, but because of capitalism, so happy Friday everyone!

— Rachel Coldicutt (@rachelcoldicutt) June 14, 2019

Updated 14 June 2019