TIBCO platform for organizational intelligence

By adding tibbr to its established software portfolio, TIBCO has now extended its range of organizational intelligence technologies. Last week I spoke with Stefan Farestam of TIBCO to discuss the present and future prospects for TIBCO customers linking these technologies together in interesting ways.

We talked about three main technology areas: Complex Event Processing (CEP), Business Process Management (BPM) and Enterprise 2.0. For TIBCO at least, these technologies are at different stages of adoption and maturity. TIBCO's CEP and BPM tools have been around for a while, and there is a fairly decent body of experience using these tools to solve business problems. Although the first wave of deployment typically uses each tool in a relatively isolated fashion, Stefan believes these technologies are slowly coming together, as customers start to combine CEP and BPM together to solve more complex business problems.

Much of the experience with CEP has been in tracking real-time operations. For example, telecommunications companies such as Vodafone can use complex event processing to monitor and control service disruptions. This is a critical business concern for these companies, as service disruptions have a strong influence on customer satisfaction and churn. CEP is also used for autodetecting various kinds of process anomalies, from manufacturing defects to fraud.

One of the interesting things about Business Process Management is that it operates at several different tempi, with different feedback loops.

• A modelling and discovery tempo, in which the essential and variable elements of the process are worked out. Oftentimes, full discovery of a complex process involves a degree of trial and error.

• An optimization and fine-tuning tempo, using business intelligence and analytics and simulation tools to refine decisions and actions, and improve business outcomes.

• An execution tempo, which applies (and possibly customizes) the process to specific cases.

The events detected by CEP can then be passed into the BPM arena, where they are used to trigger various workflows and manual processes. This is one of the ways in which CEP and BPM can be integrated.

Social software and Enterprise 2.0 can also operate at different tempi - from a rapid and goal-directed navigation of the social network within the organization to a free-ranging and unplanned exploration of business opportunities and threats. TIBCO's new product tibbr is organized around topics, allowing and encouraging people to develop and share clusters of ideas and knowledge and experience.


Curiously, the first people inside TIBCO to start using tibbr were the finance people, who used it among other things to help coordinate the flurry of activity at quarter end. (Perhaps it helped that the finance people already shared a common language and a predefined set of topics and concerns.) However, the internal use of tibbr within TIBCO has now spread to most other parts of the organization.

The organization of Enterprise 2.0 around topics appears to provide one possible way of linking with CEP and BPM. A particularly difficult or puzzling event (for example, a recurrent manufacturing problem) can become a topic for open discussion (involving many different kinds of knowledge), leading to a coordinated response. The discussion is then distilled into a resource for solving similar problems in future.

TIBCO talks a great deal about "contextually relevant information", and this provides a common theme across all of these technologies. It helps to think about the different tempi here. In the short term, what counts as "contextually relevant" is preset, enabling critical business processes and automatic controls to be operated efficiently and effectively. In the longer term, we expect a range of feedback loops capable of extending and refining what counts as "contextually relevant".

• On the one hand, weak signals can be detected and incorporated into routine business processes. Wide-ranging discussion via Enterprise 2.0 can help identify such weak signals.

• On the other hand, statistical analysis of decisions can determine how much of the available information is actually being used. Where a particular item of information appears to have no influence on business decisions, its contextual relevance might need to be reassessed.

 The adoption of Enterprise 2.0 within the enterprise raises different challenges to the adoption of CEP and BPM. One reason for this is the tricky question of critical mass. Whereas it is possible to conduct a meaningful pilot for CEP or BPM in a small part of the business, it is much harder to get a sense of how Enterprise 2.0 tools will work across the enterprise from a small pilot, and much harder to see concrete return on investment. However, many of TIBCO's customers already have an objective to implement some form of Enterprise 2.0, and the demand is simply to satisfy this objective in the most effective way.
 

---

My eBook on Organizational Intelligence is now available from LeanPub. leanpub.com/orgintelligence

Related posts: Two-Second Advantage (May 2010), Embedding Intelligence into the Business Process (November 2010)

Embedding Intelligence into the Business Process 2

#orgintelligence #entarch In my previous post, I talked about two aspects of Embedding Intelligence into the Business Process.

• Embedding business intelligence (BI) into the business process.
• Embedding Enterprise 2.0 into the business process. 

In this post, I'm going to talk about two further aspects of this.

• Embedding knowledge (content) into the business process.
• Embedding learning into the business process.

---

Embedded Knowledge Content

There are various levels at which knowledge can be embedded in a business process. Some forms of procedural knowledge can be encapsulated as static rules, which can be either written into the process (as software or bureaucratic procedure) or stored in a form that can be easily and automatically referenced by software components or knowledge workers. There is a considerable software literature on so-called business rules - see for example my review of Business Rule Concepts.

More complex forms of knowledge can be represented as models. For example, the business processes associated with operating a complex industrial process or communications network require some representation of the physical structure and processes involved, while business processes in the finance world may use economic models that help to predict market trends and risks. These models may be buried within complicated algorithms, or represented visually in dashboards and control room displays. See my post on OrgIntelligence in the Control Room.

Thirdly there is contextual knowledge - an appreciation of the specific circumstances and general trends relevant to a business decision or action. This kind of knowledge is dynamic and typically requires human mediation and interpretation, although it may be possible to codify and even automate some limited kinds of contextual knowledge. When discussing the contribution of Enterprise 2.0 to the American security services, Dennis Howlett comments that "content without context in process is meaningless". (See my post on Connecting the Dots).

In her post on The Future of HRM Software, Naomi Bloom talks about embedded intelligence that integrates the rule-based and the contextual knowledge into a software agent she calls "Naomi". She claims that embedded intelligence can achieve several things.

• It "replaces what we lost" when we reduced or eliminated the interaction between experts and the rest of the organization. (In her piece, the experts are HR professionals.)
• It improves upon human embedded intelligence by removing human error. 
• Automated embedded intelligence improves compliance to rules/policies/regulations and reduces the organization’s exposure to risk.
• Commercial Web sites (Landsend.com, Amazon.com) and social Web sites (Wikipedia) set expectations of the embedded intelligence to be found in any self service environment. 

In my post Intelligent Knowledge Management, I pointed out the important step from collaborating-in-the-work (for example shared responsibility for decisions) and collaborating-in-the-knowledge (for example, shared responsibility for collecting and interpreting intelligence, connecting the dots). I also advocated a shift of emphasis from knowledge sharing to knowledge embedding - grounding the work in the best available and critically evaluated knowledge, as well as actively seeking well-grounded knowledge to support organizational learning.

One of the ways that enterprise architects can think strategically about business capabilities and business processes is in terms of knowledge intensity - in other words, the quantity and quality of knowledge required in a given capability or process to differentiate the enterprise from its competitors. The "core" activities of an enterprise are those requiring high levels of knowledge intensity and specificity, other activities can be regarded as "peripheral" and may be commoditized or outsourced.  See my post on Ecosystem SOA, which draws on the work of Amin and Cohendet.

---

Embedded Learning

In my post on Learning by Doing, I pointed out that such characteristics as adaptability, agility, flexibility, responsiveness (supposed to be the benefits of various technologies including SOA) imply processes of adaptation and learning. So we need to ask: How do business systems (both organizational and technical) improve? Where is the learning located? What is the nature of the feedback loop?

1. The learning loop goes through the software developers. (The software development acts as a gate/brake on the learning process.)
2. A learning process is contained in the software or service. (Learning can take place in real-time, but only for things that have been explicitly anticipated in software development.)
3. The learning process is distributed across the community usage of the software or service.

In planning for organizational intelligence, we need to think about these learning processes, and how they may be accommodated in any sociotechnical system architecture. Advanced software (from SOA to Enterprise 2.0) gives us new and more flexible ways of implementing such learning processes, but only if we identify the learning requirements properly. We are going to need a business model that includes the learning capabilities as well as the operational capabilities, and an architecture that mobilizes these capabilities in a loosely coupled manner.

---

Places are still available for my Organizational Intelligence Workshop on December 8th.

Embedding Intelligence into the Business Process

Is the business process evolving from bureaucratic workflow towards some form of flexible intelligence? Some of us have been predicting this for a few years now, but there are some hopeful signs that it may finally be starting to happen.

In this post, I'm going to talk about two specific aspects of this.

• Embedding business intelligence (BI) into the business process.
• Embedding Enterprise 2.0 into the business process.

---

Embedded BI

The idea of embedded business intelligence has been around for many years. See my blog on Service-Oriented Business Intelligence from September 2005. See also my slideshare presentation.

When software vendors talk about embedded BI, they often mean embedding BI functionality in other pieces of software - for example ERP applications - to allow these applications to produce more interesting reports. There are several niche BI producers in this space, including Jaspersoft, Pentaho and Yellowfin. Brian Gentile of Jaspersoft talks about this in his recent article The BI Revolution: Business Intelligence's Future. TDWI November 10, 2010. For an article explaining the difference between Embedded BI and Integrated BI, see Execution MIH.

For BI to be embedded in the business process, we need to have an understanding of the business process that includes some cognitive task, such as a complex decision, where some business intelligence capability can be used specifically to support this cognitive task. In some cases, the aim might be to make the process faster and more efficient, but more usually the aim is to make the process more powerful and effective.

Embedded BI in this sense can also be related to intelligent event processing, where analytic capability embedded in one process can trigger automatic as well as human responses in other processes. Brian Gentle talks about this in an earlier article The BI Revolution: A New Generation of Analytic Applications. TDWI October 20 2010. 

Beyond embedding BI in the business process, we might look forward to a state in which analytics is embedded in the entire enterprise, what Tom Davenport and his colleagues call the Analytic Organization. (See my review of Competing on Analytics.) This is the proper meaning of the term Pervasive BI, which Dave Mittereder defined in 2005 as "empowering everyone in the organization, at all levels, with analytics, alerts and feedback mechanisms" (Pervasive Business Intelligence: Enhancing Key Performance Indicators Information Management Magazine, April 2005).

---

Embedded Enterprise 2.0

In her piece Time For Enterprise 2.0 To Get Enterprisey, Sandy Kemsley takes a sceptical look at the extent to which Enterprise 2.0 is supporting the core business.

"You hear great stories about social software being used to strengthen weak ties through internal social networking, or fostering social production by using a wiki for project documents, but many less stories about using social software to actually run the essential business processes."

She quotes Andrew McAfee:

[The CIOs] weren’t too worried that their people would use the tools to waste time or goof off. In fact, quite the opposite; they were concerned that the busy knowledge workers within their companies might not have enough time to participate.

And comments:

The fact that the knowledge workers had a choice of whether to participate tells me that the use of social business software is still somewhat discretionary in these companies, that is, it’s not running the core business operations; if it were, there wouldn’t be a question of participation.

It seems to me that there are two possible interpretations of McAfee's remark. Sandy's interpretation is that busy knowledge workers simply don't find time to do any Enterprise 2.0 stuff at all, and she concludes that if the business process can still work without it, then the Enterprise 2.0 stuff is discretionary. An alternative interpretation might be that the business knowledge workers don't have enough time to do enough Enterprise 2.0 stuff to get as much intelligence (requisite variety) into the business process as the business really needs. (I happen to prefer the second interpretation, but I don't know whether this is what McAfee really meant.) In other words, it could be that there a trickle of benefit rather than a decent flow.

I'm presuming that the way Enterprise 2.0 is used within the business process is to support specific cognitive tasks, such as interpreting and making sense of events, and making complex decisions. These tasks may be done by an individual knowledge worker, possibly drawing on knowledge made available by co-workers, or may be done collectively by a network of knowledge workers. The quality of sense-making and decision-making doesn't necessarily increase just because you have more people spending more time on it, but with highly complex business situations the opposite is almost certainly true - the quality will be impaired if you have too few people devoting insufficient time and attention.

But I worry a bit when technology vendors merely invoke the magic words "business process" without demonstrating any real understanding. For example, Sandy's blog links to Klint Finley's piece on Tying Enterprise 2.0 to Business Processes, or Creating New Processes for the Social Enterprise, which doesn't say anything I can recognize as being about business process; as far as I can see, it is largely about activity stream filtering as a technical solution for integrating pieces of software. Finley's piece links in turn to a Monday Musing by R "Ray" Wang which states that 

Organizations seeking a marketing edge must digest, interpret, and asses (sic) large volumes of meta data from sources such as Facebook Open Graph.

I think there may possibly be a business process implicit in there somewhere, but exactly how this business process would be supported by Enterprise 2.0 is left to the imagination. I hope "asses" isn't a Freudian slip.

To be clear, I can see how the technologies Klint and "Ray" are talking about might possibly be embedded into a sociotechnical system to support a real business process. But they aren't actually making the connection, nor are they providing any evidence that anyone else is doing so. Even Michael Idinopulos, who at least sounds as if he knows what he is talking about in The End of the Culture 2.0 Crusade? fails to provide any concrete examples. He may have seen some evidence, but he's not telling us. So (not for the first time) it's left to Tom Davenport to say something useful. In a short blogpost for HBR, he provides a couple of examples of what can be done when the social and structuring aspects of technology are combined (Want Value from Social? Add Structure).

---

Note: some of the larger software vendors have a stake in several of these areas, and are trying to integrate different product lines. For example, IBM adds predictive analytics and social networking in Cognos 10 (SearchBusinessAnalytics.com, 25 Oct 2010). Meanwhile, the niche software providers may be developing interesting partnerships and collaborations - Brian Gentle emails me with a note about the embedding of Jaspersoft within eBuilder, a Swedish provider of an end-to-end B2B suite of Cloud Supply-Chain Management Processes, to produce what they call a Strategic Management Tool.

---

Places are still available for my Organizational Intelligence Workshop on December 8th.

Are BPM professionals experts in collaboration?

@ReduxOnline posts a link to a student project How to measure the collaboration of knowledge workers in the enterprise.

1. To understand the factors supporting collaboration between knowledge workers, a questionnaire was sent to BPM professionals. The paper doesn't make clear whether they were being asked about their own personal collaboration, or about their opinions about collaboration in general. In any case, we might imagine that BPM professionals have a particular perspective on collaboration, which might distort the survey.

2. Nearly a quarter of the BPM professionals couldn't make sense of the collaboration model on which the survey was based, and were unable to answer all the questions. Instead of treating this as a sign that there might be a problem with the model, the researchers chose to exclude these from the analysis. They then argue that the remaining responses validate their model.

3. The survey doesn't measure collaboration, it measures opinions about collaboration, from a carefully selected group of knowledge workers. Perhaps not surprisingly, the opinions are pretty consistent with the kind of management literature that these knowledge workers might be expected to have read. Except that the answers about "purpose" were all over the place (which I can well imagine, given the uncertain intentions of the question), so this factor failed a statistical test (Cronbach's Alpha) and could be quietly dropped from the model.

4. The direct relation between collaboration and the performance of an enterprise is not tested, because the questionnaire did not consists of any financial questions. (It would seem that financial information is "sensitive"; collaboration itself presumably isn't.) Let's hope the students manage to extend their research to include questions about the financial situation of an enterprise, allowing them to demonstrate and explain how maturity of collaboration of knowledge workers (as perceived and understood by BPM professionals) might actually help to improve the performance of an enterprise.

5. I generally regard opinion surveys as low-grade research because they usually merely recycle received opinion. While I understand that this may be the easiest and cheapest form of research, especially for students and software industry analysts, I expect to see some acknowledgement of the potential distortion, rather than merely taking the collected opinions at face value.

And then there were more ...

Following my previous post on Consolidation: And then there were two ...

Did I give the impression that ERP was more-or-less into the end-game, once Oracle had bought PeopleSoft and Siebel? Well I am afraid I forgot about PeopleSoft co-founder and former chairman David Duffield, whose new ERP company Workday has just bought Cape Clear.

They don't give up easily, do they. Meanwhile Jan Baan (founder of Baan Software, which was bought by Invensys, then sold to SSA, and is now part of Infor Global, the third largest ERP company) is running BPM player Cordys. And of course Ross Perot founded Perot Systems after selling EDS to General Motors.

Is Bill Gates really going to be happy and fulfilled as a philanthropist? Is he going to run for president? (He'd do better than pRospero, that's for sure.) Or will he get the software itch again?

Business IT Alignment

Should IT investment be tightly coupled with business strategy? Yes of course, says the business executive. IT should be focused on delivering value according to the demands of the business.

Should business strategy be tightly coupled with IT investment? No of course not, says the business executive. That would be putting the cart before the horse. That would mean that the business strategy would have to be focused on extracting value from IT capability. That would mean we couldn't change business direction faster than the IT planning cycle. No way.

There is undoubtedly a problem in the relationship between business and IT. Many people are convinced that the solution to this problem is something called "business IT alignment". Challenged by a reader who suggests that this alignment is just more waffle, Neil Ward Dutton insists that alignment has three elements

• aligning IT investment with business strategy
• aligning IT delivery with business priorities
• aligning IT change with business change.

But the difficulty with this solution is that alignment connotes synchronization and inhibits innovation. Business and IT simply don't work to the same timetable. Alignment is symmetric - if you align A to B, this means B has to be aligned to A. That's okay if you don't ever want to change your business strategy (which many old-fashioned IT strategy planning methods assumed) or experiment with new forms of IT in advance of any business requirement. But it's hardly appropriate for the dynamic volatile world that IT vendors and consultants claim to support.

Perhaps we should be talking instead about business IT liberation - freeing business from the constraints of IT investment by decoupling the business from IT. In other words, the opposite of alignment. Isn't this what SOA is all about?

---

Discussion continues at Business IT Alignment 2 (July 2006)
See also my Architecture posts on BPM and SOA.

Activity-Based Computing

Just had an interesting demonstration from IBM Lotus of some new collaboration technology they are developing. Looking beyond the demonstration (and what IBM currently has working) there is some exciting potential for the convergence of three sets of technologies:

• CSCW (computer-supported collaborative work). Allows co-workers to collaborate around some activity. Distribution and co-development of documents related to the activity, including rich media.
  

• BPM (Business Process Management). More structured collaboration, possibly involving orchestration and workflow, as well as BAM (Business Activity Monitoring).
  

• Web 2.0. Liberal use of tags, feeds and other internet mechanisms.
  

I'm not going to go into details about IBM's current and future technology here, but here are a few examples of how this kind of technology might be used.

• A team within an organization producing a proposal for a customer. (This is a classic example, because it typically involves workers who are engaged in this class of activity as a major part of their jobs, often working on several large and complex proposals in parallel.)

• Handling a customer complaint. (Even if most complaints are relatively simple, the scope of this class of activity is much greater, potentially involving any aspect of the organization's operations. Although there may be a customer service department with primary responsibility for handling complaints, other parts of the extended organization may be pulled in as needed.)

• Solving a production problem. (A problem may be broken down into subproblems, and this may involve spawning a number of other activities. See my earlier post on BPM and SOA, which includes a discussion of the problem-solving methodology TRIZ.)

• A doctor orchestrating medical intervention for a patient - tests, drugs, physiotherapy, and so on. (The interesting aspect of this class of activity is that the collaboration typically goes outside the boundaries of the doctor's own organization. So we would need some kind of federation - for example to coordinate the activity between the clinic and the hospital.)
  

Now let me mention some of the opportunities and challenges that this kind of technology throws up.

• Balance between standardization and flexibility. We're not talking about rigidly prescriptive business processes here, but about situated processes - in other words, processes that are driven by the requirements of a particular situation. For example, although it is certainly possible to standardize some aspects of customer complaint handling - every complaint gets logged, policies for providing refunds, and so on - many complaints will require detailed investigation whose exact form depends on the content of the complaint itself. Meanwhile, management will typically want to have a standard basis for measuring the efficiency and effectiveness of such activities. It is obviously possible to define standard activity patterns, and standard services to support flexible activities - but the establishment and enforcement of these standards is a crucial aspect of activity governance.

• Model-based management. A collaborative activity may be described by an activity model or workflow model (for example using UML or BPEL for People.) It may be possible to use these models in the creation or governance of activities. It may also be possible to infer models (and therefore extract patterns) from the actual history of activities. (The demonstration I was given did not include any process visualization or inference - but there are various IBM and other Eclipse-based modeling tools with which this collaboration technology could presumably be integrated.)

• Publish/subscribe. Use of tagging and feeds potentially provides a neat way of making connections with occasional users. (Atom is a more tightly structured standard than RSS, and may be more appropriate for applications like these that go far beyond simple news.)
  

• Pull/push mechanisms. Subject-matter experts within an organization might subscribe to various tags, with the effect of identifying activities where their expertise might be relevant (thus pulling themselves into these activities). Other social networking mechanisms may also be relevant in making connections between the Expertise and the Work.
  

• Semantics. A small team may have a fairly homogeneous set of tags. As collaborations get more widely distributed, the meaning of a given tag becomes increasingly uncertain, and so the semantic value of the tags may be diluted. This problem may be tackled either top-down (ontology) or bottom-up (folksonomy) - but there are some important issues either way.

• Federation and interoperability. How do we manage collaborative activities that cross organizational boundaries? (a) If the clinic and the hospital are using separate installations of the same collaboration technology. (b) If the clinic and the hospital are using different collaboration technologies.
  

We also had a very interesting discussion on AJAX, enterprise mashups, situated software and other Web 2.0 topics, which I shall write up separately.