Tuesday, August 9, 2011

The Innovation Dilemma

"If it ain't broken, don't fix it."Sound advice, but limited to situations where "fixing it" only entails restoring past performance. In contrast, innovations entail substantive improvements over the past. Innovations are not just corrections of past mistakes, but progress towards a better future.

However, innovations often present a challenging dilemma to decision makers. Many decisions require choosing between options, one of which is both potentially better in the outcome but markedly more uncertain. In these situations the decision maker faces an "innovation dilemma."

The innovation dilemma arises in many contexts. Here are a few examples.

Technology. New and innovative technologies are often advocated because of their purported improvements on existing products or methods. However, what is new is usually less well-known and less widely tested than what is old. The range of possible adverse (or favorable) surprises of an innovative technology may exceed the range of surprise for a tried-and-true technology. The analyst who must choose between innovation and convention faces an innovation dilemma.

Investment. The economic investor faces an innovation dilemma when choosing between investing in a promising but unknown new start-up and investing in a well-known existing firm.

Auction. "Nothing ventured, nothing gained" is the motto of the risk-taker, while the risk-avoider responds: "Nothing ventured, nothing lost". The innovation dilemma is embedded in the choice between these two strategies. Consider for example the "winner's curse" in auction theory. You can make a financial bid for a valuable piece of property, which will be sold to the highest bidder. You have limited information about the other bidders and about the true value of the property. If you bid high you might win the auction but you might also pay more than the property is worth. Not bidding is risk-free because it avoids the purchase. The choice between a high bid and no bid is an innovation dilemma.

Employer decision. An employer must decide whether or not to replace a current satisfactory employee with a new candidate whose score on a standardized test was high. A high score reflects great ability. However, the score also contains a random element, so a high score may result from chance, and not reflect true ability. The innovation dilemma is embedded in the employer's choice between the current adequate employee and a high-scoring new candidate.

Natural resource exploitation. Permitting the extraction of offshore petroleum resources may be productive in terms of petroleum yield but may also present officials with significant uncertainty about environmental consequences.

Public health. Implementation of a large-scale immunization program may present policy officials with worries about uncertain side effects.

Agricultural policy. New technologies promise improved production efficiency or new consumer choices, but with uncertain benefits and costs and potential unanticipated adverse effects resulting from use of manufactured inputs such as fertilizers, pesticides, and machinery, and, more recently, genetically engineered seed varieties and information technology. (I am indebted to L. Joe Moffitt and Craig Osteen for these examples in natural resources, public health and agriculture.)

An essay like this one should - according to custom - end with a practical prescription: What to do about the innovation dilemma? You need to make a decision - a choice between options - and you face an innovation dilemma. How to choose? All I'll say is that the first step is to identify what you need to achieve from this decision. Recognizing the vast uncertainties which accompany the decision, choose the option which achieves the required outcome over the largest range of uncertain contingencies.

If you want more of an answer than that, consult your favorite decision theory (like info-gap theory, for instance).

I will conclude by drawing a parallel between the innovation dilemma and one of the oldest quandaries in political philosophy. In The Evolution of Political Thought C. Northcote Parkinson explains the historically recurring tension between freedom and equality.

Freedom. People have widely varying interests and aptitudes. Hence a society that offers broad freedom for individuals to exploit their abilities, will also develop a wide spread of wealth, accomplishment, and status. Freedom enables individuals to explore, invent, discover, and create. Freedom is the recipe for innovation. Freedom induces both uncertainty and inequality.

Equality. People have widely varying interests and aptitudes. Hence a society that strives for equality among its members can achieve this by enforcing conformity and by transferring wealth from rich to poor. The promise of a measure of equality is a guarantee of a measure of security, a personal and social safety net. Equality reduces both uncertainty and freedom.

The dilemma is that a life without freedom is hardly human, but freedom without security is the jungle. And life in the jungle, as Hobbs explained, in "solitary, poor, nasty, brutish and short".


  1. We take it for granted, nowadays, that we somehow expect progress, mostly in technology, but also in other respects. Now the concept of progress is a fairly recent phenomenon (seventeenth-eighteenth century). In antiquity there were various ways of looking at the course of history: cyclical, things go up and down until you reach a catastrophe and then everything begins again, or pessimistic: everything was better and gets worse. With Christianity people decided that this world is sad, but the world to come is perfect. There was therefore in ancient Greece and Rome no idea that science and technology are to be fostered. Only in a few technical treatises (medical, mostly) of the fifth century do you encounter a viewpoint that there is progress (technical, not social or moral).

  2. Very nice post. Just one point: The "dilemmas" you're describing are tradeoffs rather than genuine dilemmas. As you'd know, the term "dilemma" these days refers to situations where individuals' (rational) pursuit of their self-interests leads to the system failing to deliver on those interests (as in over-consumption of a common-pool resource or free-riding in the provision of a public good).

  3. it is intresting to see how these approach exsited in all fields around us. i think that surprise elements is high with human behavior rather than other fields like engineering but still we can use this Approah and build some probability model for the Uncertainity, is it possible?

  4. Probability is useful in situations of fairly structured uncertainty. In situations of ignorance and surprise we usually do not have enough information to formulate or verify a probabilistic model.