Wednesday, December 10, 2014

Have you got big data in your underwear?

Apparently, women's breasts aren't all the same. (Who knew?) True&Co. uses an algorithm based on customer feedback to recommend comfortable and flattering bras for its customers. A visitor to the website completes a questionnaire, and the website recommends some suitable bras. If the customer orders the bra, she then completes another questionnaire providing feedback on comfort and appearance. To date, over a million women have completed the questionnaire, providing 15 million data points.

@tetradian reckoned this is a great example of #bizmodel #bigdata for mass-uniqueness. But I didn't see this example the same way: I don't see anything here that turns Mass Customization into what Tom likes to call Mass Uniqueness.

Tom's favourite example of "mass uniqueness" is Picasso. I bet the algorithm couldn't find a bra for the breasts of Picasso's Demoiselles (NSFW).
Breasts of Picasso’s Demoiselle (NSFW)…

A single questionnaire, even from a million women, doesn't get into the big data league. Maybe it would when they start analysing pictures and videos of customer breasts, rather than relying on a simple questionnaire.

Or if the company were to fit sensors to its underwear, monitoring stretch during a range of activities, collecting millions of data points every minute via the Internet of Things.

Do you think I'm joking? Microsoft is working on a Smart Bra, which will monitor the mood of the wearer and detect stress. The Daily Mail suggests that this will help women to lose weight.

"To stop women reaching for the cookie jar when things hit a low, Microsoft's new prototype bra predicts when the wearer is likely to comfort eat and warns against it. The software company's high-tech undergarment features sensors in the cup pockets and side panels that detect changes in heart rate, skin temperature and stress levels - apparent precursors to overeating. All of the data is then streamed via Bluetooth to a smartphone app providing real-time 'mood-triggered eating' alerts."

Now that's what I call big data. Scary, huh?

April Joyner, Big Data: Coming Soon to Your Bra? (Fast Forward, 6 September 2014)

Jillian Goodman, Cup Size Isn’t Everything (Fast Company, October 2014)

Microsoft working on a smart bra to measure mood (BBC News, 3 December 2013)

Sadie Whitelocks, Supporting your body in more ways than one! The high-tech bra designed to stop women from comfort eating (Daily Mail, 28 November 2013)

Tom Graves, On Mass Uniqueness (23 May 2014)

Thursday, November 27, 2014

Misunderstanding CRM and Big Data

Listening to @peter_w_ryan, @markhillary and Alexey Minkevich talking about #CRM and #BigData at the Institute of Directors, sponsored by IBA Group.

Peter cites an Ovum survey showing that Customer Satisfaction is now the number one concern of management, and argues for what Ovum calls Intelligent CRM. (CA announced something under this label back in October 2000. Other products are available.)

Mark says that CRM and Big Data are widely misunderstood, which is certainly true. My own opinion is the first misunderstanding is to think CRM is about managing THE relationship with THE customer, and I completely agree with Clayton Christensen (via Sloan) that this isn't enough. What we really need to focus on is the job the customers are trying to get done when they use your product or service.

Who is good at CRM? Peter cites an example of a professor of marketing who got a personalized service at a certain chain of hotels and has been talking about it ever since. (That's a pretty good coup for the hotel, if we take the story at face value.) Mark cites the video game market, where both the console manufacturers and the large game publishers are able to collect and analyse huge quantities of consumer behaviour.

Is CRM with Big Data merely a new way of taking advantage of customers? Although most people seem oblivious to the privacy and trust risks, the Wall Street Journal this week suggested that the consumer is becoming more savvy and less susceptible to exploitative loyalty schemes and promotions. This might help to explain why Tesco, once a master of the science of retail, now seems to be faltering.

If there is a sustainable business model based on CRM and Big Data, it must surely involve using these technologies to engage intelligently, authentically and ethically with customers, rather than imagining that these technologies can provide a quick fix for stupid organizations to take advantage of compliant customers.

Related Blogs

Customer Orientation (May 2009)

The Science of Retail (April 2012)

Other Articles

Martha Mangelsdorf, Understanding your customer isn't enough (Sloan Review May 2009)

Shelly Banjo and Sara Germano, The End of the Impulse Shopper (Wall Street Journal 25 November 2014)

Intelligent CRM

AI-CRM "An intelligent CRM system with atuo-learning-tunning engine (sic), Aichain offers the most widely used open source business intelligence software in the world." Last updated March 2013

CA rolling out customer relationship management software (ComputerWorld October 2000)

IBA Group "maintains its focus on IT outsourcing that has become a strategy for many organizations seeking to improve their business processes"

Wednesday, November 05, 2014

Inside the Whale

The ethics of "pay-to-play" are being explored in the video games market. Some "free" games are funded by optional in-game purchases, such as extra weapons. There are several problems with this. Firstly, the availability of ever-more-powerful weapons might distort the game. Obviously a wealthy player could beat any chess grandmaster in the world, if the game of chess allowed you to buy an extra queen from time to time. In which case, what is the point of playing at all?

Secondly, a small minority of players are funding everyone else. In the video game world, these are known as whales. Video games often come out in multiple (overlapping) editions, and some whales can be persuaded to buy every edition. So the game developers and the majority of the players are exploiting the whales. That's fine if they can afford it, but some of the whales may be tempted to spend more than they can afford.

Some software vendors have accused industry analysts of operating a "pay-to-play" system, which allegedly favours those vendors who are willing to buy into the latest jargon (hype). And some analyst firms have a tendency to fragment any given technological marketplace into subdivisions in order to sell more "editions" of expensive reports, and to provide more differentiated opportunities for vendors to earn (or allegedly purchase) a good rating.

So if one were to produce a 2x2 matrix based on Commercial Ambition and Willingness to Invest, then the top right quadrant could be called Whales.

"If you're not familiar with how the analyst industry works, here's a very short summary. Companies brief analysts on their plans, the state of their businesses and the products they are bringing to market. Some companies pay the analyst firms to have a two-way conversation and get advice from the analysts. Then the analysts write up information-rich reports about some important business trend or another, based on all the briefings they've done. They sell those reports for hundreds or thousands of dollars to consumers wanting to benefit from all the research performed by the analysts." (Source ReadWrite Web August 2009)

Is there a pay to play problem in the analyst industry?, You told us which analyst firms were most, and least, independent (Influencer Relations, February 2014)

Tuesday, May 13, 2014

What shape is the internet (continued)?

@ironick and I have been arguing about the shape of the internet since my September 2010 post on this subject. Over the past few days, we have returned to this topic on Twitter. Nick has captured the latest tweets in his Storify piece The Shape of the Web - Database Wars Redux.

The argument was triggered by @djbressler's observation that some new browsers (including an experimental build of Chrome) were hiding the URL from the user. This is a reflection of the fact that users increasingly type "Amazon" into the browser rather than "" let alone "". Presumably, hiding the URL will further encourage this trend.

Google and other search engines appear to benefit from this in two ways. Firstly, it increases the already heavy dependence of the ordinary internet user on the search engine. And secondly, every time an internet user navigates via search rather than via URL or hyperlink, the search engine gets another opportunity to present some advertising, as well as collecting more information about that user.

Obviously, Google itself depends on URLs and hyperlinks. As Nick points out, Google still relies on links to construct its index, and still uses a version of the original PageRank algorithm to influence what you see when you search for a given term. But indexing and search ranking are only loosely coupled to one another.

And nowadays, the search order is not solely determined by PageRank. Instead, the search order is increasingly influenced by browsing behaviour - of others as well as our own. If you ignore the first two items, click briefly on the third item, and then immediately return to Google to look at the fourth item, Google may conclude that the first three items weren't very relevant to you. In other words, this counts as a "vote" against those items.

Meanwhile, Google only had exclusive rights to the original PageRank patent (which belongs to Stanford University) until 2011.

Obviously Google is not completely open about these algorithms, because it is perpetually at war with SEO and spammers who want to get some commercial advantage by "gaming" the system. So there is a degree of speculation involved in working out what exactly Google is up to. Sometimes Google merely seems to appeal to the lowest common denominator, as David Auerbach suggests in his review of Metafilter search results ("Deranked"). However, it is beyond speculation that Google's behaviour has become increasingly sophisticated over the past decade, and that what we see is increasingly "personalized".

Nick accuses me of "confusing the use of behavior IN the ranking algorithm itself with using behavior to verify the quality of the algorithm". However, there is some evidence that Google initially trials new factors in parallel with the existing algorithm, before integrating these factors into the algorithm itself. (See for example, Google Panda.) In any case, the total behaviour of Google can be thought of in terms of the collective intelligence of human brains AND algorithmic software, and it may not be possible for outsider observers to be exactly sure where the boundary lies at any point in time. (We can detect "momentum", but not "position".)

Obviously URLs are not going to disappear entirely. For my part, I have always made an effort to use links and bookmarks rather than pander to the commercial interests and cognitive distortion of search engines. I don't think this undermines my general point - that the Internet-in-use (based on majority habits) is taking on a different shape. Obviously it is still possible to use the Internet in a disciplined and self-conscious manner, which Nick (always) and I (sometimes) practise, but the fact that this requires effort and intelligence makes it likely that it will never become mainstream.

In the long-term, Google may face a paradox. If people stop using URLs, then Google's ability to index and rank pages across the internet might possibly be compromised. But I'm sure that the clever people at Google have thought of this paradox, and already have a cunning plan.

Meanwhile, the internet (as experienced by ordinary users) is gradually becoming less web-shaped and more star-shaped, with your favourite search engine or social network at the centre. (Please note the word "gradually".)


David Auerbach, Deranked - Why has Google forsaken MetaFilter? (Slate May 2014)
Bill Slawski, The New PageRank, Same as the Old PageRank? (March 2012)
Daniel Sour, It Knows (LRB October 2011)

Related posts

What shape is the internet (September 2010)
What shape is your intranet (May 2014)

Updated  17 May 2014

Bring Your Own Expectations

Once upon a time, there was a clear separation between Work and Home. This separation has been undermined by two phenomena.

1. Working at Home - in other words, allowing work to invade the home environment
2. Bring Your Own Device (BYOD) - in other words, allowing personal devices to invade the work environment

In this post, I want to talk about a third phenomenon, perhaps more invisible but no less important. Bring Your Own Expectations means that we have all become accustomed to getting what we want from the Internet, and therefore expect to get the same things (or "affordances") from corporate systems and platforms.

One of the most obvious gaps between our expectations and corporate reality is the failure of search. The Internet has an uncanny knack of guessing what we want, and there are strong commercial incentives for Amazon, Facebook, Google and the rest to improve their "mind-reading" capabilities.

In comparison, your company intranet is simply not in the same league, and therefore cannot anticipate your needs in the same way. Some people see this as merely a technical lack, to be addressed by some functionality inside the company firewall that roughly resembles the way Google worked ten years ago. But this is far more than a mere technical shortcoming.

And search is just one difference. There are also expectations about interoperability. For example, do we expect to use one network for linking with colleagues and customers, and a different network for linking with friends and family? Most people are still learning how to manage these different worlds without getting muddled - for example, people who automatically put kisses onto private messages may find themselves carrying this habit into corporate communications. Maybe sometimes our expectations lead us astray.

Some service providers (including notoriously Facebook) insist that you have a single identity for personal and business use. Other service providers accept that people may wish to have two or more accounts, in order to keep personal and business use separate, and are happy to design premium services largely for the business user. A good example of this is DropBox for Business, which allows multiple accounts (e.g. a business account and a personal account) to be synchronized to the same computer. However, people will still expect to have at least as much affordance in the business sphere as in the personal sphere, and will be unhappy if their employer provides (for example) corporate file-sharing services that are not as good as (say) DropBox. (Other file sharing services are available.)

Related Posts

BYOD Bring Your Own Device (Feb 2012)
On Working At Home (March 2014)
What Shape is Your Intranet (May 2014)

Updated 5 November 2014

Saturday, May 10, 2014

What shape is your Intranet?

@djbressler tells us he is work­ing on a thought-piece about the bifur­ca­tion of the intranets from the Inter­net. In the meantime, in a post called Burying the URL (May 2014), he comments on an experimental build of Chrome, which continues the trend of hiding the URL and encouraging people to use search instead. Obviously it benefits Google when people rely exclusively on search. But it's not just Google's Chrome that is doing this; Firefox and Mobile Safari are also going in this direction.

In my post What shape is the internet (September 2010), I said that shifting the emphasis from URL hotlinks to search undermines the idea of the internet's being web-shaped. This point is also made in a post by @apike, referenced by David and also called Burying the URL (April 2014).

URLs are the essence. They make hypertext hyper. The term “web” is no accident – it refers to this explicitly.
See also an excellent Twitter debate following @apike's tweet.

When David talks about bifurcation, he means that "enter­prise IT is diverg­ing enter­prise tech­nol­ogy from con­sumer tech­nol­ogy in a way that’s cre­at­ing two irrec­on­cil­able branches of tech­nol­ogy". He observes that most company intranets have a pretty lousy search facility.

But most company intranets have pretty lousy cross-linking as well. They are mostly just pdf graveyards stuffed with documents of indeterminate pedigree, which people are often reluctant to waste time searching (even if the search facility were better) because they don't expect to find anything of value.

Actually, you can't always find what you are looking for on the Internet either, and that has a lot to do with the limitations of search, but there are enough amusing distractions to conceal this fact. Surely we don't want our company intranets to copy the internet too closely?

And remember that the data revealing Enron's problems were cheerfully displayed on the Enron website. But nobody important had bothered to look at these documents properly. (Actually, a bunch of students had analysed them years previously and concluded that Enron was bankrupt. They probably got low marks for that assignment!)

There is an increasingly common belief that the tech­nol­ogy used inside com­pa­nies should work the same way as outside, should provide the same "affordance". This is not Bring Your Own Device but Bring Your Own Paradigm Expectations. I guess I should work on a thought piece about this.

Related posts

What shape is the internet (September 2010)
Bring Your Own Expectations (May 2014)

See also Steven Poole, The pdf graveyards can only expect an increase in their undead populations (Guardian 9 May 2014)

Saturday, April 26, 2014

Does Big Data Release Information Energy?

@michael_saylor of #Microstrategy says that the Information Revolution is about harnessing "information energy" (The Mobile Wave, p 221). He describes information as a kind of fuel that generates "decision motion", driving people - and machines - to make a decision and take a course of action.

We already know that putting twice as much fuel into a vehicle doesn't make it twice as fast or twice as reliable. (Indeed, aeroplanes sometimes dump fuel to enable a safer landing.) But Saylor explains that information energy is not the same as physical energy.

1. Information energy doesn't follow conservation laws. Information can be created, consumed repeatedly, but never depleted or destroyed. (Unless it is lost or forgotten.)

2. Whereas physical energy is additive, the energy content of information is exponential.

3. The value of information depends on its use, and who is using it.

Let's look at his example.

"Total wheat production for a single year is valuable information; but total wheat production for ten years, combined ten years of rainfall data and ten years of fertilizer represents thirty times more data droplets, but probably contains one hundred times more information energy, because it shows trends and correlations that will drive a greater number of decisions." (pp 221-2).

In other words, thirty times as much data produces a hundred times more information. He doesn't say this extra information MAY drive more decisions, he says it WILL drive more decisions. In other words, the Information Revolution (and our increasing reliance on tools such as Microstrategy's products) is a historical inevitability.

But is it really true that more data produces more information in this exponential way? In practice, there is a depreciation effect for historical or remote data, because an accumulation of small changes in working practices and technologies can make direct comparison misleading or impossible. So even if the farmer had twenty years' worth of data, or shared data from thousands of other farmers, it would not necessarily help her to make better decisions. Five years' data might be almost as good as ten years'.

Data is moving faster than ever before; we're also storing and processing more and more of it. But that doesn't mean we're just hoarding data, says Duncan Ross, director of data sciences at Teradata, "The pace of change of markets generally is so rapid that it doesn't make sense to retain information for more than a few years." (Charles Arthur, Tech giants may be huge, but nothing matches big data, Guardian 23 August 2013)

According to Saylor, the key to releasing information energy is mobile technology.

"The shocking thing about information is not how much there is, but how inaccessible it is despite the immense value it represents. ... Mobile computing puts information energy in hands of individuals during all waking hours and everywhere they are." (p 224)

What kind of decisions does Saylor imagine the farmer needs to make while sitting on a tractor or milking the cows? Obvious it would be useful to get an early warning of some emerging problem - for example an outbreak of disease further down the valley, or possible contamination of a batch of feed or fertilizer at the factory. But complex information needs interpretation, and most decisions require serious reflection, not instant reaction.

So it is not clear that providing instant access to large quantities of information is going to improve the quality of decision-making. And giving people twice as much information often leads to further procrastination. Surely the challenge for Microstrategy is to help people deal with information overload, not just add to it?

Furthermore, as I said in my post Tablets and Hyperactivity (Feb 2013), being "always on" means that you never have long enough to think through something difficult before you are interrupted by another event. There is always another email to attend to, there is always something happening on Twitter or Facebook, and mobile devices encourage and reinforce this kind of hyperactivity.

Saylor concludes that "the acid of technology etches away the unnecessary" (p 237). If only this were true.

Related posts

Service-Oriented Business Intelligence (September 2005)
On The True Nature of Knowledge (April 2014)

Updated 19 June 2014

Thursday, April 24, 2014

Predictive Analytics for the Smart Consumer?

#CW500 If merchants can use predictive analytics to get more out of the customer, why can't the customer use predictive analytics to get more out of the merchant?

In December 2012, I reported on a subscription-based service from, which predicted future retail price changes (based on retailers' past behaviour) and encouraged its members to use these predictions to optimize the timing of key purchases.

Many retailers have fairly regular patterns of seasonal price changes and promotions, which are designed to maximize the lifetime profitability of a product. This is particularly important for fashion goods and high-tech, which tend to have a high initial price and a low clearance price. However, if customers (with the help of advisory services such as start to game these price changes, then profit optimization becomes a lot harder to calculate. So this kind of advisory service represents a significant threat to retail profitability.

In September 2013, was acquired by eBay and effectively closed down. “This is an exciting opportunity to bring Decide’s expertise in data and predictive analytics to the worldwide commerce leader and empower over 25 million eBay sellers,” said Mike Fridgen, CEO of “We believe teaming up with eBay allows us to realize our mission of leveling the playing field in commerce.” (eBay 6 September 2013, Geekwire 6 September 2013)

In other words, taking the advantage away from the customers and giving it back to the sellers.

However, other customer-side predictive services may be still available, including (for Gasoline) and Kayak (for air travel).

Predictive Showrooming Dec 2012

Tricia Duryee, Says It Will Accurately Predict Prices or Your Money Back (All Things D, 19 April 2012)

Thorin Klosowski, Kayak Adds Price Forecasting to Predict Price Drops and Increases (Lifehacker, 15 Jan 2013)