From Networked BI to Collaborative BI

Back in September 2005, I commented on some material by MicroStrategy identifying Five Types of Business Intelligence. I arranged these five types into a 2x2 matrix, and commented on the fact that the top right quadrant was then empty. 

 
The Cloud BI and analytics vendor Birst has now produced a similar matrix to explain what it is calling Networked BI, placing it in the top right quadrant. Gartner has been talking about Mode 1 (conventional) and Mode 2 (self-service) approaches to BI, so Birst is calling this Mode 3.

While there are some important technological advances and enablers in the Mode 3 quadrant, I also see it as a move towards Collaborative BI, which is about the collective ability of the organization to design experiments, to generate analytical insight, to interpret results, and to mobilize action and improvement. This means not only sharing the data, but also sharing the insight and the actioning of the insight. Thus we are not only driving data and analytics to the edge of the organization, but also developing the collective intelligence of the organization to use data and analytics in an agile yet joined-up way.

I first mentioned Collaborative BI on my blog during 2005, and discussed it further in my article for the CBDI Journal in October 2005. The concept started to gather momentum a few years later, thanks to Gartner, which predicted the development of collaborative decision-making in 2009, as well as some interesting work by Wayne Eckerson. Also around this time, there were some promising developments by a few BI vendors, including arcplan and TIBCO. But internet searches for the concept are dominated by material between 2009 and 2012, and things seem to have gone quiet recently.

---

Previous posts in this series

Service-Oriented Business Intelligence (September 2005)
From Business Intelligence to Organizational Intelligence (May 2009)
TIBCO Platform for Organizational Intelligence (March 2011)


Other sources

Gartner Reveals Five Business Intelligence Predictions for 2009 and Beyond (Gartner, January 2009). Dave Linthicum, Let's See How Gartner is Doing (ebizQ, May 2009)

Chris Middleton, Business Intelligence: Collaborative Decision-Making (Computer Weekly, July 2009)

Ian Bertram, Collaborative Decision-Making Platforms (Gartner 2011)

Wayne Eckerson, Collaborative Business Intelligence: Optimizing the Process of Making Decisions (April 2012)

Monique Morgan, Collaborative BI: Today and Tomorrow (arcplan, April 2012)
Tiemo Winterkamp, Top 5 Collaborative BI Solution Criteria (arcplan, April 2012)

Cliff Saran, Prepare for two modes of business intelligence, says Gartner (Computer Weekly, March 2015)

The Future of BI is Networked (Birst, March 2016)


Updated 21 April 2016 (image corrected)

Predictive and Real-Time Analytics

I shall be chairing the @UNICOMSeminars Data Analytics conference next week. Exploring the Business Value of Predictive and Real-Time Analytics (London, 2 December 2015)

A lot of the obvious applications of real-time analytics are in fraud detection and predictive maintenance. I shall be talking about some of the things I’ve been doing recently in the retail and consumer sector, using rich consumer data to drive real-time personalized engagement with the consumer across multiple touchpoints. We have been exploring ways to combine real-time analysis of the consumer’s current state (e.g. current location, what products they are currently looking at, readiness to buy, etc.) with a rich understanding of what one might call the consumer’s “purchasing genes” – for example, do they like to spend a long time reviewing alternative products before purchasing, do they like to wait for a special offer or voucher before buying, or on the other hand do they like to be the first in their social network to have a given product. This is a lot more complex than simply putting them into a fixed number of “consumer segments”.

Based on this analysis, it is possible to select an appropriate “next action” – for example, selecting the appropriate banner to display to the consumer when visiting the website, or the right topic of conversation for a human customer services agent.

Thus predictive analytics are helping retail as it moves from omnichannel commerce (which joins up the buying transaction between the online and the physical world) to omnichannel engagement and marketing (which joins up all aspect of the relationship with the consumer).

Omnichannel Commerce (Systems of Record) joins up the buying transaction between the online and the physical world

Omnichannel Marketing (Systems of Engagement) joins up all aspects of the relationship with the consumer

Given the large volumes of data involved, and the reliance on legacy systems to produce and process the data, we are not yet seeing this analysis being completely done in real-time. However, there are some critical factors that have to be done in real-time. For example, as soon as the consumer buys something, our clients want to stop trying to sell it, and move to a post-sales scenario. (In comparison, even the great Google is still showing me advertisements based on what I was browsing three weeks ago. Fail!)

Over the next couple of years, as the technology gets better, the data scientists get even smarter, and the marketing people get more sophisticated, we may expect an increasing proportion of the analysis to be done in real-time, using machine learning as well as more sophisticated analytics tools.

Real-Time Personalization from Richard Veryard

Towards the Internet of Underthings

#WearableTech #InternetOfThings Once upon a time, the wires in an undergarment merely provided structural support. Now, people may have all sorts of wires and wireless devices hidden under their clothing. Here are some interesting examples.

• The Foxleaf Bra delivers cancer-fighting drugs through the wearer's skin.
• An aunt’s death led Kemisola Bolarinwa to develop a wearable device that can pick up Nigeria’s most common cancer much earlier.
  
• The @tweetingbra reminds women to examine themselves. (?)
• The Lumo Lift helps improve posture through app-enabled coaching.
• Various manufacturers (including Clothing+, OMsignal and SmartLife) produce health vests and sportswear packed with monitors to track your heart rate, breathing rate and the amount of calories you've burnt.

We are now encouraged to account for everything we do: footsteps, heartbeats, posture. Until recently this kind of micro-attention to oneself was regarded as slightly obsessional, nowadays it seems to be perfectly normal. And of course these data are collected, and sent to the cloud, and turned into someone else's big data. (Good luck with those privacy settings, by the way.)

If a device is classed as a medical device, it will be subject to various forms of accreditation and regulation. For this reason, many device makers will be careful to avoid any specific medical claims, but devices that offer some health advice are considered a borderline area.

Another borderline area is hi-tech underpants that protect men from the evil rays allegedly produced by all those wireless devices. Especially the radiation from mobile phones. (Including the Bluetooth that links your underwear to your smartphone.) One brand of underpants that claims to use a mesh of pure silver to create a Faraday cage around the genitals has been banned by the UK Advertising Standards Authority from making any medical claims.

Or maybe you could just switch the whole lot off.

---

The Wearable Medical Device in Your Future…Is Now! (Marketing Research Association, 28 April 2015)

Jennie Agg, The hi-tech bra that helps you beat breast cancer - and other clothes that can treat or prevent illness (Daily Mail, 10 March 2015)

Valentine Benjamin, Can a bra detect breast cancer? This Nigerian entrepreneur thinks so (Guardian, 9 Aug 2023)

Sarah Blackman, Student designs cancer-fighting bra (Lingerie Insight, 10 Feb 2015)

Britta O'Boyle, SmartLife clothing claims to make sure you never miss a beat (Pocket-Lint, 12 March 2015) 

Rob Crilly, Hi-tech pants "protect sperm from phone waves" (Telegraph 22 October 2014)

Julie Papanek, How Wearable Startups Can Win Big In The Medical Industry (TechCrunch, 19 Feb 2015)

Hannah Jane Parkinson, Lumo Lift review: posture-tracking gadget is a straight shooter (Guardian, 14 November 2014) 

Helen Popkin, Tweeting bra exposed: Genuine support or publicity lift? (NBC News 25 October 2013)

Meera Senthilingam, How a high-tech bra could be your next doctor (CNN, 11 May 2015)

Brendan Seibel, High-Tech Underwear for Adventurous Geeks (21 April 2010)

Mark Sweney, Hi-tech underwear advert banned (Guardian 13 August 2014)

Dan Sung, World Cancer Day - The Real Wonderbra (Wearable, 14 Feb 2015)  


Related Posts Have you got big data in your underwear (December 2014)

Have you got big data in your underwear?

Apparently, women's breasts aren't all the same. (Who knew?) True+Co. uses an algorithm based on customer feedback to recommend comfortable and flattering bras for its customers. A visitor to the website completes a questionnaire, and the website recommends some suitable bras. If the customer orders the bra, she then completes another questionnaire providing feedback on comfort and appearance. To date, over a million women have completed the questionnaire, providing 15 million data points.

@tetradian reckoned this is a great example of #bizmodel #bigdata for mass-uniqueness. But I didn't see this example the same way: I don't see anything here that turns Mass Customization into what Tom likes to call Mass Uniqueness.

Tom's favourite example of "mass uniqueness" is Picasso. I bet the algorithm couldn't find a bra for the breasts of Picasso's Demoiselles (NSFW).

Breasts of Picasso’s Demoiselle (NSFW)…

A single questionnaire, even from a million women, doesn't get into the big data league. Maybe it would when they start analysing pictures and videos of customer breasts, rather than relying on a simple questionnaire.

Or if the company were to fit sensors to its underwear, monitoring stretch during a range of activities, collecting millions of data points every minute via the Internet of Things.

Do you think I'm joking? Microsoft is working on a Smart Bra, which will monitor the mood of the wearer and detect stress. The Daily Mail suggests that this will help women to lose weight.

To stop women reaching for the cookie jar when things hit a low, Microsoft's new prototype bra predicts when the wearer is likely to comfort eat and warns against it. The software company's high-tech undergarment features sensors in the cup pockets and side panels that detect changes in heart rate, skin temperature and stress levels - apparent precursors to overeating. All of the data is then streamed via Bluetooth to a smartphone app providing real-time mood-triggered eating alerts.

Now that's what I call big data. Scary, huh?

---

Valentine Benjamin, Can a bra detect breast cancer? This Nigerian entrepreneur thinks so (Guardian, 9 Aug 2023)

Jillian Goodman, Cup Size Isn’t Everything (Fast Company, October 2014)

Tom Graves, On Mass Uniqueness (23 May 2014)

April Joyner, Big Data: Coming Soon to Your Bra? (Fast Forward, 6 September 2014)

Hayley Krischer, The underappreciated artistry of the professional bra fitter (Guardian 4 June 2015)

Sadie Whitelocks, Supporting your body in more ways than one! The high-tech bra designed to stop women from comfort eating (Daily Mail, 28 November 2013)

Microsoft working on a smart bra to measure mood (BBC News, 3 December 2013)


See also Towards the Internet of Underthings (November 2015), Weaving in three dimensions (November 2015), On the Ethics of Technologically Mediated Nudge (May 2019), Trial by Ordeal (July 2019)

 
Links added 16 July 2020, 9 August 2023

Misunderstanding CRM and Big Data

Listening to @peter_w_ryan, @markhillary and Alexey Minkevich talking about #CRM and #BigData at the Institute of Directors, sponsored by IBA Group.

Peter cites an Ovum survey showing that Customer Satisfaction is now the number one concern of management, and argues for what Ovum calls Intelligent CRM. (CA announced something under this label back in October 2000. Other products are available.)

Mark says that CRM and Big Data are widely misunderstood, which is certainly true. My own opinion is the first misunderstanding is to think CRM is about managing THE relationship with THE customer, and I completely agree with Clayton Christensen (via Sloan) that this isn't enough. What we really need to focus on is the job the customers are trying to get done when they use your product or service.

Who is good at CRM? Peter cites an example of a professor of marketing who got a personalized service at a certain chain of hotels and has been talking about it ever since. (That's a pretty good coup for the hotel, if we take the story at face value.) Mark cites the video game market, where both the console manufacturers and the large game publishers are able to collect and analyse huge quantities of consumer behaviour.

Is CRM with Big Data merely a new way of taking advantage of customers? Although most people seem oblivious to the privacy and trust risks, the Wall Street Journal this week suggested that the consumer is becoming more savvy and less susceptible to exploitative loyalty schemes and promotions. This might help to explain why Tesco, once a master of the science of retail, now seems to be faltering.

If there is a sustainable business model based on CRM and Big Data, it must surely involve using these technologies to engage intelligently, authentically and ethically with customers, rather than imagining that these technologies can provide a quick fix for stupid organizations to take advantage of compliant customers.

---

Related Blogs

Customer Orientation (May 2009)

The Science of Retail (April 2012)

Other Articles

Martha Mangelsdorf, Understanding your customer isn't enough (Sloan Review May 2009)

Shelly Banjo and Sara Germano, The End of the Impulse Shopper (Wall Street Journal 25 November 2014)

Intelligent CRM

AI-CRM "An intelligent CRM system with atuo-learning-tunning engine (sic), Aichain offers the most widely used open source business intelligence software in the world." Last updated March 2013

CA rolling out customer relationship management software (ComputerWorld October 2000)

IBA Group "maintains its focus on IT outsourcing that has become a strategy for many organizations seeking to improve their business processes"

Does Big Data Release Information Energy?

@michael_saylor of #MicroStrategy says that the Information Revolution is about harnessing "information energy" (The Mobile Wave, p 221). He describes information as a kind of fuel that generates "decision motion", driving people - and machines - to make a decision and take a course of action.

We already know that putting twice as much fuel into a vehicle doesn't make it twice as fast or twice as reliable. (Indeed, aeroplanes sometimes dump fuel to enable a safer landing.) But Saylor explains that information energy is not the same as physical energy.

1. Information energy doesn't follow conservation laws. Information can be created, consumed repeatedly, but never depleted or destroyed. (Unless it is lost or forgotten.)

2. Whereas physical energy is additive, the energy content of information is exponential.

3. The value of information depends on its use, and who is using it.


Let's look at his example.

"Total wheat production for a single year is valuable information; but total wheat production for ten years, combined ten years of rainfall data and ten years of fertilizer represents thirty times more data droplets, but probably contains one hundred times more information energy, because it shows trends and correlations that will drive a greater number of decisions." (pp 221-2).

In other words, thirty times as much data produces a hundred times more information. He doesn't say this extra information MAY drive more decisions, he says it WILL drive more decisions. In other words, the Information Revolution (and our increasing reliance on tools such as MicroStrategy's products) is a historical inevitability.

But is it really true that more data produces more information in this exponential way? In practice, there is a depreciation effect for historical or remote data, because an accumulation of small changes in working practices and technologies can make direct comparison misleading or impossible. So even if the farmer had twenty years' worth of data, or shared data from thousands of other farmers, it would not necessarily help her to make better decisions. Five years' data might be almost as good as ten years'.

Data is moving faster than ever before; we're also storing and processing more and more of it. But that doesn't mean we're just hoarding data, says Duncan Ross, director of data sciences at Teradata, "The pace of change of markets generally is so rapid that it doesn't make sense to retain information for more than a few years." (Charles Arthur, Tech giants may be huge, but nothing matches big data, Guardian 23 August 2013)

According to Saylor, the key to releasing information energy is mobile technology.

"The shocking thing about information is not how much there is, but how inaccessible it is despite the immense value it represents. ... Mobile computing puts information energy in hands of individuals during all waking hours and everywhere they are." (p 224)

What kind of decisions does Saylor imagine the farmer needs to make while sitting on a tractor or milking the cows? Obvious it would be useful to get an early warning of some emerging problem - for example an outbreak of disease further down the valley, or possible contamination of a batch of feed or fertilizer at the factory. But complex information needs interpretation, and most decisions require serious reflection, not instant reaction.

So it is not clear that providing instant access to large quantities of information is going to improve the quality of decision-making. And giving people twice as much information often leads to further procrastination. Surely the challenge for MicroStrategy is to help people deal with information overload, not just add to it?

Furthermore, as I said in my post Tablets and Hyperactivity (Feb 2013), being "always on" means that you never have long enough to think through something difficult before you are interrupted by another event. There is always another email to attend to, there is always something happening on Twitter or Facebook, and mobile devices encourage and reinforce this kind of hyperactivity.

Saylor concludes that "the acid of technology etches away the unnecessary" (p 237). If only this were true.

---

Related posts

Service-Oriented Business Intelligence (September 2005)
On The True Nature of Knowledge (April 2014)


Updated 19 June 2014

Predictive Analytics for the Smart Consumer?

#CW500 If merchants can use predictive analytics to get more out of the customer, why can't the customer use predictive analytics to get more out of the merchant?

In December 2012, I reported on a subscription-based service from decide.com, which predicted future retail price changes (based on retailers' past behaviour) and encouraged its members to use these predictions to optimize the timing of key purchases.

Many retailers have fairly regular patterns of seasonal price changes and promotions, which are designed to maximize the lifetime profitability of a product. This is particularly important for fashion goods and high-tech, which tend to have a high initial price and a low clearance price. However, if customers (with the help of advisory services such as decide.com) start to game these price changes, then profit optimization becomes a lot harder to calculate. So this kind of advisory service represents a significant threat to retail profitability.

In September 2013, decide.com was acquired by eBay and effectively closed down. “This is an exciting opportunity to bring Decide’s expertise in data and predictive analytics to the worldwide commerce leader and empower over 25 million eBay sellers,” said Mike Fridgen, CEO of Decide.com. “We believe teaming up with eBay allows us to realize our mission of leveling the playing field in commerce.” (eBay 6 September 2013, Geekwire 6 September 2013)

In other words, taking the advantage away from the customers and giving it back to the sellers.

However, other customer-side predictive services may be still available, including GasPredictor.com (for Gasoline) and Kayak (for air travel).

---

Predictive Showrooming Dec 2012

Tricia Duryee, Decide.com Says It Will Accurately Predict Prices or Your Money Back (All Things D, 19 April 2012)

Thorin Klosowski, Kayak Adds Price Forecasting to Predict Price Drops and Increases (Lifehacker, 15 Jan 2013)

Using Analytics Correctly

@gcharlton quotes a survey from @dbdsearch claiming that 80% of online retailers are using @GoogleAnalytics incorrectly (October 2012), via @FreshNick @hayden30.


Clearly Google wants online retailers to use all the features of the Google Analytics platform, which entails integrating with various other Google products and services (e.g. Google Adwords) as well as implementing all the necessary tracking codes and cookies according to Google's requirements. Any online retailer that fails to conform to Google's requirements is deemed to be using the platform incorrectly.

But what does "incorrectly" mean? Not doing what Google thinks you should be doing? Since when has Google been the ultimate arbiter of correct action?

We have been here many times before. There is often a significant gap between the designed product (how its designers expect it to be used) and the product-in-use (what the users actually do with the product). A designed product may have a number of sophisticated features that most users never get around to using, perhaps never actually need. On the other hand, the users trying to do a real job of work often display remarkable ingenuity in getting the stupid product to do something much more interesting than it was designed to do.

And sometimes there is a considerable delay until users discover the more sophisticated features. To cite a historical example, most early users of Lotus Notes used it as a substitute for technologies they already had, before they started to appreciate what it had really been designed for.

So there may be many ways people could learn to use Google Analytics better. As @haydens30 says, "there are basic best practice things that a lot of sites don't do - these are easy wins for any consultant".

And there may be many ways Google itself could make Analytics better and easier to use. In announcing some UI improvements yesterday, Nikhil Roy of the Google Analytics Team said "We hope you find these improvements useful and always feel free to let us know how we can make Analytics even more usable for you to get the information you need to take action faster."

What, we have to tell them? Don't they already know?
 

Opening the Black Box: Analytics and Admissions

@peteyMIT via @EthanZ explains how technology is changing the university admissions process.

When kids apply to university in the USA, it is becoming increasingly common to include a link with supplementary information about the applicant - for example a project tumblr, a YouTube video, a Flickr album of artwork. The links are typically coded to track visitors, giving the applicant some idea about the level of interest the universities are showing. Chris Peterson finds this an uncomfortable experience: "As admissions officers, we are accustomed to reading applications; now, applications are reading us. ... Applicants are now armed with unprecedented insight into the processes that decide their fate."

There are several problems with this. Applicants and their parents may be misled by the tracking signals collected by these digital supplements, which may yield an entirely false picture of the university process. And yet applicants may attempt to use these signals as evidence that an application has not been properly considered. Even if the university attempts to block the analytics, this may still send the wrong message. (The absence of a signal is still a signal.)

In the past, analytics were a tool used by large organizations to monitor and control their customers. We are now seeing analytic platforms that seem to allow customers to monitor and control large organizations. Large organizations now need to understand how much information they are exposing to these platforms, and what conclusions their customers may draw. We can expect similar examples to appear in many other sectors.


Chris Peterson, Opening the Black Box: Analytics and Admissions (Chronicle of Higher Education, January 2013)

Updated 25 June 2015

OrgIntelligence - Are Better Tools the Answer?

Information Gathering

Managers spend up to two hours a day searching for information, and more than 50 percent of the information they obtain has no value to them.  In addition, only half of all managers believe their companies do a good job in governing information distribution or have established adequate processes to determine what data each part of an organization needs.

Source: Accenture January 2007

The average interaction worker spends an estimated 28 percent of the workweek managing e-mail and nearly 20 percent looking for internal information or tracking down colleagues who can help with specific tasks.

Source: McKinsey July 2012

See also HBR Blog August 2012

Knowledge Management

Traditional knowledge management has failed to address the problem of knowledge worker productivity. Tools that have been developed in KM focused on information management and do not support many of the key knowledge work processes. Knowledge workers have therefore adpated the email client to suit their needs. It has become the most successful knowledge work tool because it combines personal control with personalisability and integrates communication.

Source: Martin Roell, August 2004

Big Data

By 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions.

Source: McKinsey May 2011

Big data can create big value. But like all the big-data predecessors – i.e., databases, data warehousing, data mining, data analytics and business intelligence – you need to know what you’re looking for, why you’re looking for it, what’s it worth to you, and how will you take advantage of it BEFORE you start. Otherwise, big data will just be a big waste of money.

Source: Silver Peak Systems, May 2012

Social Media

Social media is addictive. And if you’re not too careful, it can seriously eat into your productivity.

Source: MediaBistro All-Twitter October 2012

---

Places are still available on my Organizational Intelligence Workshop (Feb 1st).