Software vendors have hidden motives

Gartner analyst Dennis Gaughan made some important announcements in Australia recently.

• Microsoft, Oracle, IBM, and SAP have hidden motives.
• Microsoft mainly wants to protect Windows and Office.
• Oracle products don't really work well together.
• IBM wants to take over your IT strategy.
• SAP confuses customers with pricing.

[Source: Business Insider 21 November 2011]

 A cynical view of Gartner would be that such opinions would be suppressed because Microsoft, IBM, Oracle, and SAP are clients of Gartner research. For example, @greenm3 (Dave Ohara) wonders why Gartner would want to spin in this particular way [greenm3]. For my part, I don't think I can recall a major industry analysis firm going on the record so baldly, and all credit to Dennis Gaughan for his frankness.

I'm wondering what these comments imply for the traditional Gartner criteria of "vision" and "ability to execute". How can we separate the commercial and technological capabilities and characteristics of these vendors, and can we always assume that such vendors mobilize what vision and ability-to-execute they may possess in the best interests of their customers? Let's hope Gartner will now pursue this line of analysis further.

Black Swan Blindness

In my post Black Swans and Complex System Failure, I talked about the architectural implications of some recent disasters, including the Gulf of Mexico oil spillage in 2010 and the partial melt-down in Japanese nuclear reactors following the tsunami in 2011. Both of these disasters involved something that isn't supposed to happen: the simultaneous failure of multiple fail-safe mechanisms.

A new study by Oxford University and McKinsey finds a similar phenomenon in technology investment, where large IT projects may experience spiralling costs as a result of multiple problems occurring simultaneously. According to the researchers, this is up to twenty times more frequent than traditional risk modelling techniques would expect, with one in six large IT projects going over budget by an average of over 200%. Researchers refer to the tendency to disregard rare but high-impact problems/risks as black swan blindness.

As an example, Professor Bent Flyvbjerg cites the collapse of Auto Windscreens, which went into administration in February following a disastrous attempt to implement a new IT system. "Black swans often start as purely software issues. But then several things can happen at the same time - economic downturn, financial difficulties - which compound the risk," he explained.

Professor Flyvbjerg has coined the term Black Swan Management, which currently merits its own Wikipedia page. Simon Moore (author of Strategic Project Portfolio Management) questions whether it is appropriate to use the term "black swan" for something that occurs with a one in six probability, but supports Flyvbjerg's conclusion that when projects go wrong they can go extremely wrong.

Flyvbjerg makes five fairly bland recommendations for avoiding IT project failure, including recruiting a "master builder". Some people may interpret this as an endorsement of the large IT service firms, but these firms have been responsible for some of the most extravagent failures. Is there any evidence that master builders are any more immune from "black swan blindness" than anyone else? Indeed, as a Scandinavian, Flyvbjerg will hardly need reminding of Ibsen's portrayal of madness in the play of the same name.

---

'Black swans' busting IT budgets (BBC News, 26 August 2011)

Bent Flyvbjerg and Alexander Budzier, Why Your IT Project May Be Riskier than You Think (Harvard Business Review, September 2011, pp. 601-603)

Natasha Lomas, Five ways to stop your IT projects spiralling out of control and overbudget (Silicon.com, 22 August 2011) (pdf)

Brenda Michelson, Complexity, Outliers and the Truth on IT Project Failure (HP Input-Output, 31 Aug 2011)

Simon Moore, Black Swans In Project Management (August 25, 2011)

Ethics of Risk in Public Sector IT

@tonyrcollins via @glynmoody and @Mark_Antony asks Should winning bidders tell if they suspect a new contract is undeliverable? (8 June 2011) and raises some excellent ethical points about public sector procurement.

One of the functions of good journalism is to hold people and organizations to account. Tony fishes out a speech given in 2004 by Sir Christopher Bland, then chairman of BT, in which he acknowledged incomplete success in previous ventures, and admitted the extraordinary challenges involved in the NPfIT, for which BT had just won three contracts then valued at over £2bn.

There is obviously a difference between something's being extremely difficult and its being impossible. BT executives can fairly claim that they were always open about the chance that it was going to be difficult, and that they didn't know for sure that it was going to be impossible. But at the same time, there is an asymmetry of information here - the supplier is presumably in a better position to assess certain classes of risk than the customer. (Meanwhile, there may be other classes of risk that the customer should know more about than the supplier.)

In my opinion, the ethical issues here are not to do with deliberate concealment of known facts, but of misleading or inadequate assessment of shared risk. The key word in Tony's headline is the word "suspect". So what are the ethics of doubt?

TIBCO platform for organizational intelligence

By adding tibbr to its established software portfolio, TIBCO has now extended its range of organizational intelligence technologies. Last week I spoke with Stefan Farestam of TIBCO to discuss the present and future prospects for TIBCO customers linking these technologies together in interesting ways.

We talked about three main technology areas: Complex Event Processing (CEP), Business Process Management (BPM) and Enterprise 2.0. For TIBCO at least, these technologies are at different stages of adoption and maturity. TIBCO's CEP and BPM tools have been around for a while, and there is a fairly decent body of experience using these tools to solve business problems. Although the first wave of deployment typically uses each tool in a relatively isolated fashion, Stefan believes these technologies are slowly coming together, as customers start to combine CEP and BPM together to solve more complex business problems.

Much of the experience with CEP has been in tracking real-time operations. For example, telecommunications companies such as Vodafone can use complex event processing to monitor and control service disruptions. This is a critical business concern for these companies, as service disruptions have a strong influence on customer satisfaction and churn. CEP is also used for autodetecting various kinds of process anomalies, from manufacturing defects to fraud.

One of the interesting things about Business Process Management is that it operates at several different tempi, with different feedback loops.

• A modelling and discovery tempo, in which the essential and variable elements of the process are worked out. Oftentimes, full discovery of a complex process involves a degree of trial and error.

• An optimization and fine-tuning tempo, using business intelligence and analytics and simulation tools to refine decisions and actions, and improve business outcomes.

• An execution tempo, which applies (and possibly customizes) the process to specific cases.

The events detected by CEP can then be passed into the BPM arena, where they are used to trigger various workflows and manual processes. This is one of the ways in which CEP and BPM can be integrated.

Social software and Enterprise 2.0 can also operate at different tempi - from a rapid and goal-directed navigation of the social network within the organization to a free-ranging and unplanned exploration of business opportunities and threats. TIBCO's new product tibbr is organized around topics, allowing and encouraging people to develop and share clusters of ideas and knowledge and experience.


Curiously, the first people inside TIBCO to start using tibbr were the finance people, who used it among other things to help coordinate the flurry of activity at quarter end. (Perhaps it helped that the finance people already shared a common language and a predefined set of topics and concerns.) However, the internal use of tibbr within TIBCO has now spread to most other parts of the organization.

The organization of Enterprise 2.0 around topics appears to provide one possible way of linking with CEP and BPM. A particularly difficult or puzzling event (for example, a recurrent manufacturing problem) can become a topic for open discussion (involving many different kinds of knowledge), leading to a coordinated response. The discussion is then distilled into a resource for solving similar problems in future.

TIBCO talks a great deal about "contextually relevant information", and this provides a common theme across all of these technologies. It helps to think about the different tempi here. In the short term, what counts as "contextually relevant" is preset, enabling critical business processes and automatic controls to be operated efficiently and effectively. In the longer term, we expect a range of feedback loops capable of extending and refining what counts as "contextually relevant".

• On the one hand, weak signals can be detected and incorporated into routine business processes. Wide-ranging discussion via Enterprise 2.0 can help identify such weak signals.

• On the other hand, statistical analysis of decisions can determine how much of the available information is actually being used. Where a particular item of information appears to have no influence on business decisions, its contextual relevance might need to be reassessed.

 The adoption of Enterprise 2.0 within the enterprise raises different challenges to the adoption of CEP and BPM. One reason for this is the tricky question of critical mass. Whereas it is possible to conduct a meaningful pilot for CEP or BPM in a small part of the business, it is much harder to get a sense of how Enterprise 2.0 tools will work across the enterprise from a small pilot, and much harder to see concrete return on investment. However, many of TIBCO's customers already have an objective to implement some form of Enterprise 2.0, and the demand is simply to satisfy this objective in the most effective way.
 

---

 book now  Organizational Intelligence Workshop (14 April 2011)
 see also  Organizational Intelligence Portal

IT analysis and trust

@mkrigsman asks "Trust is the currency that matters most. How many analysts / bloggers deserve it?"

@markhillary replies "surely in the same way as a journalist is trusted, by earning it"

@mkrigsman is particularly concerned about those who write about IT failure. (I'm not sure why he singles out that topic, but I note that the concern arose during a conversation with @benioff, boss of Salesforce.) "When someone writes on IT failures ask "What's their angle?". Usually sensationalism, currying favor, or threatening a vendor." When challenged about his own angle by @njames, @mkrigsman replies "I want to expose *why* projects fail, so we understand magnitude of the problem and can improve."

Trust is clearly a difficult issue for software industry analysts. Unfortunately, Michael's answer to Nigel's challenge cannot prove that he doesn't have a hidden agenda, because the untrustworthy are often just as able as the trustworthy to produce a plausible cover story. If we trust Michael it's not because he can answer the challenge but because of his track record.

We also need to ask - trusted by whom. Software companies might prefer industry analysts to be compliant and predictable, but intelligent software users might regard such analysts as being insufficiently independent. Who would you trust to tell you about Microsoft's new platform -  someone who is always pro-Microsoft, someone who is always anti-Microsoft, or someone who has a track record of making both positive and negative comments about Microsoft and its competitors?

Of course, this comment doesn't only apply to industry analysts. Robert Scoble, when he worked for Microsoft, made a point of distancing himself from the party line, and he therefore commanded a different kind of attention and respect than did Bill Gates or Steve Ballmer.

From a simplistic software industry perspective, an analyst who talks about IT success might be regarded as a friend, whereas an analyst who talks about IT failure is potentially an enemy. (This might explain Marc Benioff's wish to challenge the hidden agenda of the latter.) While many software and service companies might adopt the from-failure-to-success rhetoric - "the best way to avoid the risk of failure is to buy our software and hire our consultants" - this is not ideal from a sales perspective.

Mark Hillary appeals to a journalistic ethic, which would presumably include things like balance and transparency. But balance is not always appreciated by those with most at stake. In the past, I have written technology reports on new products, which I regarded as generally positive with a few small caveats. (I don't generally waste my time writing about products that are no good.) But the vendors concerned have often regarded my remarks as highly critical. (Fortunately, this over-sensitivity on the part of software companies is now changing, thanks in part to social media, and companies now understand that a robust debate can be just as beneficial as a highly controlled one-way marketing exercise.)

From a narrow software industry perspective, a trustworthy industry analyst is one who satisfies Simon Cameron's definition of an honest politician - "one who, when he is bought, will stay bought". But from a broader perspective, we should surely prefer to trust those industry analysts with independently critical mind, unafraid to ask awkward questions and publish the answers.

With the large industry analysis firms, the question of trust shifts from personal integrity to corporate integrity. The sales pitch for these firms depends not just on isolated flashes of insight from individual analysts, but on the collaborative intelligence of a community of analysts. Corporate integrity depends not just on transparency about the relationship between the work paid for by software vendors and the independent research consumed by CIOs, but also on a coherent and robust research methodology adopted consistently across the firm, typically supported by an apparatus of surveys and structured questionnaires and checklists and spreadsheets. However, there is a potential disconnect between the routine processing of supposedly objective raw data (this product with this market share in this geography in this time period) and the generation of useful interpretation and opinion, which is where the analytical magic and subjectivity comes in. One example of this magic, Gartner's Magic Quadrant, has been challenged in the courts; Gartner's defence has been that MQ represented opinion rather than fact. (See my post The Magic Sorting Hat is Innocent, Okay?) And the complicated relationship between fact and opinion, and the transparency of reasoning and evidence, is surely relevant to the level of trust that can be invested by different stakeholders in such analyses.

By the way, why am I writing about software industry analysis? Obviously, because I want to expose *why* analysis fails, so we understand magnitude of the problem and can improve. How can software industry analysis deliver greater levels of intelligence and value to the software industry as a whole?

Jeopardy and Risk

@Forrester's Andras Cser notes the victory of IBM's Watson computer in a TV quiz game, and asks How Can You Capitalize On This In Risk And Fraud Management?

In his short blogpost, Cser doesn't offer an answer to this question. He merely makes one assertion and one prediction.

Firstly he asserts an easy and superficial connection between the game of Jeopardy and the profession of security, based on "the complexity, amount of unstructured background information, and the real-time need to make decisions." Based on this connection, he makes a bold prediction on behalf of Forrester.

"Forrester predicts that the same levels of Watson's sophistication will appear in pattern recognition in fraud management and data protection. If Watson can answer a Jeopardy riddle in real time, it will certainly be able to find patterns of data loss, clustering security incidents, and events, and find root causes of them. Mitigation and/or removal of those root causes will be easy, compared to identifying them."

As this is presented as a corporate prediction rather than merely a personal opinion, I'm assuming that this has gone through some kind of internal peer review, and is based on an analytical reasoning process supported by detailed discussions with the IBM team responsible for Watson. I'm assuming Forrester has a robust model of decision-making that justifies Cser's confidence that the Jeopardy victory can be easily translated into the fraud management and data protection domain within the current generation of technology. (Note that the prediction refers to what Watson will be able to do, not what some future computer might be able to do.)

For my part, I have not yet had the opportunity to talk with the IBM team and congratulate them on their victory, but there are some important questions to explore. I think one of the most interesting elements of the Watson victory is not the complexity - which other commentators such as Paul Miller of Engadget have downplayed - but the apparent ability to outwit the other competitors. This ability may well be relevant to a more agile and intelligent approach to security, but that's a long way from the simplistic connection identified by Cser. Meanwhile, I look forward to seeing the evidence that Watson is capable of analysing root causes, which would be a lot harder than winning at Jeopardy.

---

Paul Miller, Watson wins it all, humans still can do some other cool things (Engadget 16 Feb 2011)
IBM's Watson supercomputer crowned Jeopardy king (BBC News 17 Feb 2011)

Technological Identity and Difference

Can we trace any link between the Apple Newton and the Apple iPad? When can we regard a technology as "the same again"?

When Apple discontinued its Newton hand-held computer in 1998, one possible explanation for this decision was that it had been "an idea ahead of its time".

Apple's surprise decision to discontinue its Newton hand-held computer ends one of the computer industry's boldest experiments. CEO Steve Jobs' decision halts further development of the Newton technology, which includes the hand-held MessagePad and the portable eMate. An earlier plan to spin off Newton as a separate entity was reversed by Jobs. Many observers interpreted Jobs' move at the time as assigning a role to Newton in Apple's evolving strategy. Apple instead will devote resources to its mainstream Macintosh computer line. Newton has won critical praise for its newer versions, but the product failed to overcome negative publicity on its handwriting recognition flaws. Competitors, such as the PalmPilot device, surpassed Newton in the market. Apple has sold about 200,000 Newtons since 1993, compared to more than one million PalmPilots sold since 1996, according to analysts.

Source: Jim Carlton, Apple drops Newton, an idea ahead of its time (The Wall Street Journal Western Edition, 1998)

It would not be hard to produce one list of the similarities between the Newton and iPad, and another list of the differences. Some people might regard the similarities as more important, and argue that the essential idea was the same. Other people might regard the differences as more important, and argue that the iPad was essentially new, with merely a distant family connection to the Newton.

For many purposes, it doesn't really matter how we talk about this relationship, and some writers may flipflop erratically between identity and difference. But if we wish to understand more deeply how technological ideas develop and evolve and spread, we need a more rigorous basis for talking about technological identity and difference. What are the essential characteristics that we must pay attention to?

Can an idea that was once ahead of its time ever return, or is it always going to be transmogrified by our intervening experience? Can we step into the same technological river twice?