Monday, March 01, 2010

Technology Intelligence

I'm going to start this blog entry with a bold statement - the purpose of industry analysis is intelligence.
  • providing intelligence about technology, relevant to both supply side and demand side - identifying and explaining present and future opportunities and threats
  • supporting intelligent reasoning about technology - making sense of technological complexity, supporting a broad range of technical decisions (not just planning and procurement)
  • closing the intelligence loop - knowledge sharing, memory and learning.
In my work on organizational intelligence, I identify six aspects of collective intelligence. The following table shows how this framework could apply to technology intelligence, and evaluates the contribution of industry analysts.

General Intelligence
Technology Intelligence
Current Best Practice
Perception and monitoring Scanning Analysts get most of their understanding of technological developments from vendors - typically either from a marketing perspective or from an R&D perspective. Predictions of technology adoption are based on a range of indirect information gathering mechanisms, including opinion surveys and internet buzz.
Sense-making Sense-making and standardization Analysts, vendors, consultancies and others try to establish frameworks and terminology, and many analyst firms appear to regard this as their primary task. However the struggle for hegemony between rival narratives sometimes produces more heat than light.
Reasoning Planning and choice Some analyst firms offer summarized product and vendor evaluations, such as the Magic Sorting Hat, that are commonly used (despite smallprint health warnings) as simple short-cut devices for technology selection.
Memory Collective experience There are many superficial case studies, but very little systematic measurement and benchmarking.
Learning Learning There may be isolated pockets of process improvement, but not much collective learning-from-experience, and no visible mechanism for improving the reliability of information gathering or the robustness of standard frameworks.
Communication and Collaboration Sharing Some anecdotal experience is shared through user communities, especially user groups for specific products, but apart from occasional speaking engagements, industry analysts do not typically play a leading role in these communities.

Overall, there are some useful things that industry analysts are currently doing, but as this table shows, there are some important feedback loops missing, and the general lack of decent metrics and benchmarks has always been a major inhibitor for technology intelligence. (I have made some attempts to address this lack in the past, but any kind of benchmarking initiative only works if you can get enough organizations to join in.)

If you go to Wikipedia for clear and independent insight on any of the topics covered by IT industry analysts, you will probably be disappointed. Most articles are either hopelessly one-sided, or bloated and contradictory, with self-appointed experts each trying to promote themselves and their opinions. We might think this is the result of a deliberate plot to devalue open-source insight, but more likely this is merely a symptom of the collective confusion about most of these topics.

Clearly there are some industry analysts doing excellent work, going way beyond the kind of "best practice" identified in this quick summary. Please let me know of any relevant activity that I may have missed.



@aleksb6 wonders why I "feel the urge to make up new terms for something that is well known and understood". But I'm certainly not the only person talking about industry analysis as a form of intelligence.

"Analysts help technology buyers and sellers understand the market environment in which both groups operate. This intelligence helps technology vendors determine the strategies and features that are important to buyers. Likewise, analysts guide enterprise buyers through the challenging process of procuring large, expensive systems. When this system works properly, analysts provide a valuable service." Michael Krigsman


  1. I would dearly like to see the industry analyst community (well treating the community as a whole is always dangerous, but bear with me here, please) do historical analysis on itself. We see predictions of market sizes, adoption rates, technology futures. But rarely do we see a scorecard trumpeting the historical accuracy of these predictions.

    Of course who would actually pay for such work? (sounds like a Masters' thesis in communications to me - especially as the research material is available in many University libraries.

    I just think it would be fascinating and maybe sharpen up the analysis we see.

  2. Any research in this area would be interesting, and I should be delighted if any masters student anywhere in the world wants to have a serious attempt at this, but what I'd really like to see is some independent body providing regular benchmarking and audit. Is this a job for the SEI or ESI?