Opinion is recursive. I have always been wary of technical research based on opinion surveys, because most technical opinions are already based on second-hand information. How are we to interpret the fact that 75% of IT managers believe that platform X is flexible, or that vendor Y is attacking vendor Z, or that the most significant success factor is P? How many of these opinions are based on first-hand experience, and how much on reading what other people think – or what other people want them to think.
(The internet amplifies this effect. Google typically finds you the information that other people have already found; blogs and wikis often circulate received opinion without critical engagement; vested interests are often imperceptible behind a complex information supply chain).
There has recently been a vigorous debate between Lotus Domino and its admirers on the one hand, and the Radicati Group. John Vaughan argues that the Radicati Group analysis is based on out-of-date opinion; Lotus is now under new leadership, and the opinions collected by Radicati don't reflect this.
I haven't looked at the rights and wrongs of this particular debate, but I want to comment on some general principles of software industry analysis.
Clearly there is often a delay between a real change and the perceived change. Outsiders don't change their opinions just because there is a new management team with some fine-sounding words. And even when the new management has started to achieve something, it may take a while before this disseminates through received opinion.
Given that there is always a huge legacy of out-of-date opinion, the challenge with opinion surveys is to detect and interpret shifts in opinion, rather than simply report the current distribution of opinion.
Obviously the vendor marketing departments care about the prevailing opinion, because this is what they are selling into. Purchasers care about prevailing opinion for a different reason: because they don't want to look stupid buying a technology that nobody else wants. But in some cases, the ideal scenario for a purchaser is to buy just before everyone else buys. You may be able to negotiate deep discounts and lots of free support from a vendor who is hungry for revenue and for customer success stories. Then if everyone follows your lead, you may appear very smart indeed.
Just like in the stock market, good decisions are based on a shrewd estimate of tomorrow's prevailing opinion. Momentum only works in the short term. And if we analysts (whether stock analysts or technology analysts) simply reflect momentum, we aren't doing our job.
Related posts: Industry Analysis by Survey (July 2009), On Practices and Pitfalls (September 2009), On the Transparency of Algorithms (October 2016)
No comments:
Post a Comment