Businesses are challenged by ever increasing degrees of complexity and volumes of data. Simply tidying up is an inadequate response. No matter how many filing cabinets you have, it’s never enough, and making use of information gets no easier. But, no worries: business processes, to the rescue! We’ll a) identify participants, b) flows of information and transactions and c) codify those. This is great! We can ensure that the right information is at the right place, at the right time. But wait! It gets better! We can automate those processes! We can use IT to route information, coordinate work, execute transactions, and make those (now visible) processes way more efficient! Cool! Business transactions will flow smoothly from customers and suppliers, to systems and workers in our organisation, and back. Our coordination costs will drop, causing our transaction costs to drop, and we’ll pocket more money.

This is great stuff.

Well, OK. Except it doesn’t work.

The real world is so much more complex than we thought, when we started trying to model it. [1]

We can’t seem to keep up.

We’ve got all this stuff in place, and now all it seems we do all day is manage exceptions to our processes. And that’s now become a new cost of doing business -- one that’s not cheap. Clay Shirky once said:

“Process is an embedded reaction to prior stupidity. We are often glad of this, of course; it explains a lot of what's good about the world. Our knives come equipped with handles,” he said, “and ... our programs include dialog boxes that say 'Are you sure you want to casually delete the last 3 hours of work?', all because of lessons learned from prior stupidity. But. But not all stupidity is amenable to deflection by process, and even when it is, the overhead created by process is often not worth the savings in deflected stupidity. Stupidity is frequently a one-off, and a process designed to deflect it within an organization actually ends up embedding it as a negative shape. Like the outline of Wile E. Coyote just after he is catapaulted through a wall, making everyone fill out The Form Designed to Keep You From Doing The Stupid Thing That One Guy Did Three Years Ago actually emphasizes the sense memoryof that stupid thing within the group. CAUTION: The beverage you are about to enjoy is extremely hot.”

Riffing off of Shirky's remark, Ross Mayfield wrote a piece called "The Death Of Process" almost three years ago -- a piece which achieved, and continues to enjoy, a certain degree of notoriety. He said:

Organizations are trapped in a spiral of declining innovation led by the false promise of efficiency. Workers are given firm guidelines and are trained to only draw within them. Managers have the false belief engineered process and hoarding information is a substitute for good leadership. Processes fail and silos persist despite dysfunctional matrices. Executives are so far removed from exceptions and objections that all they get are carefully packaged reports of good news and numbers that reveal the bad when it's too late.

John Seely Brown and John Hagel point out that while 95% of IT investment goes to support business process (to drive down costs), most employee time isn't spent on process -- but exceptions to process. Further, competitive advantage comes from how we innovate in handling exceptions. When something fails, informed and empowered employees turn to their social network. The informal network, or heterarchy, where most business gets done

.

There was a good bit of pushback, at the time, some of it from some fairly notorious voices themselves. The basic gist of the contrary response was that process is an inevitable, and absolutely necessary part of every conceivable economic interaction. I don't think that's incorrect, and so long as one interprets Ross's rant as being a screed about process, in the abstract, then I think that argument applies. But I'm not sure that this is what Ross actually meant, and regardless, it still leaves us with a rather large elephant in the room.

The problem is not business processes. The problem is trying to automate business processes.

We are more efficient than before, but we’re disappointed nevertheless. Yes, our coordination costs are lower than they were with ad hoc and / or manual processes. But now we want more! We want to keep enjoying these improvements in efficiency and productivity, but we want the creativity and innovativeness back, which we are somehow certain that we've lost (even if we can't quite prove it, such things being famously hard to measure). We want more efficiency, more productivity, more, more, more! This is (at least, in part) what Sig is talking about in his brilliant post on "Barely Repeatable Processes". And the overall meme that Ross codified continues to percolate, years later.

Fair or not, reasonable or not, this stuff hasn’t lived up to its hype. What now?

OK. Back to the drawing board, then! If it's not working, we must just not be doing enough of it. We’ll just add each of those exceptions to the process, as we discover them. Over time, the process will grow to handle everything. Right?

Well, no. This approach seems to lead to process models that are too complex. We can't reason about them anymore, we can't test them anymore, and therefore, we’re starting to get nervous about making any changes to them at all -- we don’t know what side effects might emerge. What’s going on?

There are multiple, interlocked problems at work here. For one thing, what many business people failed to understand is that modeling business processes is programming. Automated or not (but obviously, even more so in the automated case), creating a codification of a business process (whether graphical or textual) is a cognitive act that in no way differs from that of writing a computer program. Frankly, vendor hype colluded with, and in many cases, created this misunderstanding, and continues to feed and care for it.

The even more important thing to understand here is that, in many cases, what the business analysts then proceeded to attempt to codify was tacit knowledge. Add into that mix the fact that the environment in which the process runs is constantly changing (evolving) and the process, ideally, should be able to adapt to that (this is effectively what is happening when we start trying to successively add newfound exceptions to our processes.) Well, guess what? You know what a codification of tacit knowledge -- an algorithm of any kind that could take tacit knowledge into account -- would likely be? One that could evolve along with its environment? Artificial intelligence. The process in question would be a “thinking machine”.

There are lots of different approaches to solving these sorts of problems, some more promising than others. And it's likely that these efforts will lead to steady, if incremental, improvements in the ability of automated processes to model ever more of the overall business process domain space. Nevertheless, insofar as what the business is trying to achieve is artificial intelligence, this stuff will continue to fail to achieve that. What many people thought BPM automation could achieve just isn’t possible. Why? Because real business processes involve tasks that can only be solved with one tool -- sentience. Even more frustrating, the most valuable bits in any given process are always the ones that need this pesky sentience stuff. But there is no HAL 9000 yet -- artificial intelligence continues to elude us. Machines aren’t sentient. So what’s the alternative?

People. Human beings. Kind of obvious, isn’t it? Within any given value chain, at any given step in any given process, there is one particular human being best suited to handle whatever that context is. This is an oversimplification, of course -- maybe there are two people, or a group of people, or whatever. You get the point.

That doesn’t mean that business processes are useless. It also doesn’t mean that automating them was a bad idea -- think of the efficiencies we’ve gained as a result of doing so. No need to toss out that bathwater, let alone any babies. People like to talk about “the pendulum swings back” and so forth, and on one level, this is clearly about that. The perceived loss of flexibility and innovativeness associated with over-enthusiastic attempts to make assembly lines out of every process has resulted in a backlash -- the pendulum that swung towards “industrialization” swings back towards “human interactions”. Except it doesn’t, really. I’ve never liked the pendulum analogy, because, strictly speaking, it implies that we oscillate back and forth between two fixed points -- when it “swings back”, we end up back where we were before. That’s clearly not what happens. Instead, I like to imagine it as a pendulum moving in three directions: somebody, let’s call him Foucalt, is carrying it, whilst walking forward (yes, I am a progressive). Now, we have to assume that the pendulum is still swinging of its own accord, but this is Foucalt, after all, and he’s good with pendulums. The point being, because he’s walking with it, along the arrow of time, when the pendulum swings “back”, it arrives at a place further along towards wherever it is Foucalt is going then it was before. That is true here as well. We’re not oscillating back to where we were before we started automating business processes: that would not only be stupid, it’s -- upon reflection -- clearly impossible.

So how do we move forward with Foucalt? How can we improve the situation? Consider what happens when a case in our call centre is too weird for the process to handle. What then? Well, the call centre operator will typically toss the case to the back office, in the hope that somebody there can handle it. Thus, that exception handling cost we mentioned earlier is characterised by trying to find the right person for a given task... It’s a question of navigating a social network.

"But wait!" cries our BPM modeller. "Hey , that’s routing! That’s great! It is just a modelling problem. We just have to refine the models until we can express that routing problem. Hmm, OK, so we need better ways of categorising our workers (more categories), and those categories need to be constantly updated to reflect new and changing circumstances, and, umm, and..."

Well, no. We’re still just running to stand still. Don’t underestimate routing problems, for one thing. People cope with those kinds of problems more efficiently than machines can. Moreover, finding the right person is only half the game -- it’s only one part of what that sentient participant does (or, more precisely, could do).

The other part is collaboration. People can collaborate with each other to solve problems in remarkably efficient ways -- if they’re allowed to. Static business processes, however, often don’t allow (let alone encourage) collaboration.

If we’re going to evolve at a faster pace than our competitors, we can either invent HAL 9000, or make it easy and rewarding to find and collaborate with one other. I know what I'd bet on.

And, as it happens, finding and collaborating with each other is exactly what this “Enterprise 2.0” stuff is all about. The term was originally popularised by Andrew McAfee, a professor at the Harvard Business School, and he came up with a mnemonic for it: SLATES. Over time, the term has been refined somewhat, a fact best seen in the FLATNESS mnemonic coined by Dion Hinchliffe, an analyst and enterprise architect. “Social networking software in the enterprise” is probably the simplest possible way of expressing the concept, although it’s important not to disregard the attributes represented by SLATES (and FLATNESS). So what is “social networking software”? In a nutshell, it is software that helps people find, and collaborate with, other people. Sound familiar?

How do blogs fit that description? Or wikis? What about stuff like instant messaging? Or email? Since this stuff is still sorting itself out, it probably makes sense not to obsess about the definition. The pundits will tell you that IM isn’t E2, for example (or at least, some of them will), essentially because the information exchanged in an IM transaction isn’t a) publicly visible, and b) persistent. I think this is a good point, but also hair splitting. The biggest reason why IM and email aren't social networking tools all by themselves is that they don’t actively support you in finding the right person -- they can only assist you in collaborating with that person after the fact. But that’s nothing to be sneezed at, and as part of the overall mix of functionality, these kinds of point-to-point tools are still incredibly useful. The key is whether a tool helps people find and / or collaborate with each other. Blogs and wikis, for example, are conversational content repositories -- they definitely pass our simple test.

So what can we do to make use of this idea? We need to build Enterprise 2.0 capabilities -- social networking software functionality -- into our business processes. Ideally, we need to build that functionality into the same software we use to automate and manage our business processes -- we want to enable people to find each other, and collaborate with each other, all within the “flow” of their normal tasks. This is an important point, and we can’t afford to underestimate it -- the really big win for us will come when we can enable people to collaborate such that tasks are completed with less friction: lower coordination costs. That will produce measurable gains, reductions in cost, and make it relatively easy to measure (and achieve) ROI.

That’s also why it’s important that we find a way to bake this into a single, coherent user interface. BPM tasks, blogging, wikis, user profile searches, tagging, linking -- all in one place. If there’s one aspect of Enterprise 2.0 that can’t be emphasised enough, it’s this -- the barriers to entry (usage) have to be as low as conceivably possible. Enterprise 2.0 is a consequence of, and informed by, the dizzying pace of innovation on the consumer web. Social processes have to be as easy (and fun!) to use as consumer software on the open Internet.

There are anecdotal indications that other benefits may emerge. One clear example is concurrency -- the execution of different pieces of a given task in parallel. IT systems struggle with this, and people are better at it. The geeks may howl in protest at that, but in fact, it’s true. Despite all of the focus on concurrency in IT systems, their capabilities remain laughable compared to “simple” human intelligence. Ad-hoc arrangements, made as part of collaborative work, can achieve efficiencies that algorithmic solutions cannot -- lacking a HAL 9000. And if done via software baked into the overall solution, this can be achieved with no loss of accountability or process visibility.

The overall meme of Enterprise 2.0 also encompasses a lot more than just social processes. The use of social networking software can (and probably will) accelerate the change of an organisation from a hierarchical, command-and-control structure to a networked one. Increased collaboration also correlates tightly with other benefits, such as increased innovation. Social processes are just one element among many -- one answer to one question, not the answer to all questions.

But it’s probably not unreasonable to argue that social processes are the “killer app” for Enterprise 2.0.

And there's an important point lurking here -- I’ve talked a bit about the potential to regain some degree of creativity and innovation in our business processes. But that’s not an integral part of the “killer app” message. The “killer app” message is much simpler -- with social software in the mix, exception handling (inevitably a human labour intensive task) can be made even more efficient. Coordination costs can be reduced because social software can help us to find the “right” person faster, with less effort, and collaboration can enable us to solve problems faster and with less friction. The resulting reduction in coordination costs is complemented by a reduction in the cost of process maintenance -- our processes can be leaner, they don’t have to attempt to model every conceivable exception, etc. Taken together, these effects can lead to a measurable reduction in overall transaction costs -- and where there’s measurability, there is ROI. Any other (potential) benefits will be emergent.

For me, the penny really dropped at the Enterprise 2.0 conference in Boston in June. Ross spoke there of embedding social software (primarily wikis, of course) "in the flow" of existing business process. He was talking about an idea originally expressed by Michael Idinopulos, in this post. And as I heard Ross talk about this, I finally "got it" -- this is the low hanging fruit. This is where the enterprise and social software can meet with the fastest, clearest ROI. That's non-trivial, first off because this is an issue that has increasing urgency. And, of course, as Lee Bryant pointed out, finding intersections of the existing enterprise and social software may well be the most important thing we can work on in our time.

By adding social networking functionality to existing BPM systems, enterprises can get the benefits of Enterprise 2.0 with clear, measurable business results. Reducing coordination costs within existing business processes is smack in the crosshairs of social networking software. Social processes -- the combination of BPM and Enterprise 2.0 will be greater than the sum of those parts.

And until HAL 9000 goes online, that combination is likely to provide a massive competitive advantage to somebody. Why not you?

[1] {geekMoment} Interestingly, Alan Turing himself didn’t seem to believe that computation could be limited to algorithmic problem solving. He wrote a paper in 1936 in which he suggested that the model now commonly known as a “Turing machine” had “limited power”, and proposed something he called “choice machines”, an abstraction that extended a Turing machine to interactive computation. But people like John Von Neumann and Martin Davis ignored this, and since the 60’s, computer science has been dominated by the concept that computation is entirely mathematical. On a very real (if abstract) level, this is just the ancient battle between rationalism and empiricism. Rational systems are inherently “closed”, empirical systems “open”. If Gödel, quantum mechanics, and the like, have shown us anything, it’s that the “real world” is unfortunately rather open ended. {geekMoment}