Thursday, January 07, 2010

Should we take OWASP seriously?

Another stimulating discussion with @mcgoverntheory (James McGovern) about the ongoing OWASP project to identify the Top Ten Security Risks. I see no reason to change my previous opinion , which is that such lists are fundamentally misconceived.

As I've explained before (in this blog and elsewhere), I think the objectives of the list are muddled; I regard the methodology for producing the list as insufficiently rigorous; and I think it highly likely that the list will be widely used not as a precursor to a serious threat analysis but as a lazy substitute for it; so I just can't see that a Top Ten list is a good idea for anyone.

@mcgoverntheory replies "Many contributors to the top ten agreed that top ten lists as a concept are flawed. Its all about helping others move needle." Yes, but does it actually achieve any positive outcome? Show me.

@mcgoverntheory adds "Flawed concepts are propagated all the time. It's called marketing". But is it really the role of OWASP to be a marketing organization?

@mcgoverntheory continues "Everyone knows that Top X lists aren't meant to be complete nor necessarily measurable. Its about simple understanding". Well maybe everyone knows, but what matters is whether and how they act upon that knowledge.

@mcgoverntheory admits that "Sadly, most enterprises start and stop with awareness". Maybe so, but why should OWASP pander to this tendency?

And if OWASP is focusing its efforts on publicizing material that many contributors agree to be flawed, why on earth should industry analysts take OWASP seriously? Does OWASP want to be taken seriously?

Maybe it doesn't. @mcgoverntheory asks "What lift would analysts provide to OWASP? No products to sell and therefore we won't show up in quadrants or hype docs."

Of course, that depends what kind of industry analysis we are talking about. Some so-called industry analysis firms seem to do little more than reprocess and amplify the efforts of the software industry marketing departments, putting favoured products and vendors into a Magic Sorting Hat. Or they write like a theatre critic who gets invited to the previews, always finds something positive to say about the latest production, which can then be quoted on the play's website.

But I hope OWASP isn't the kind of organization that only wants analysis on its own terms, and understands that the value of industry analysis comes from the different perspective an analyst should be able to offer. In which case, I am happy to talk.

No comments: