Categories

Category Theory

The Process of Theory Building

I started working at The Clayton Christensen Institute and my job is to help develop the theory of disruptive innovation.

In order to do this I need to understand at least two concepts:

  • The process of theory building
  • Disruptive innovation theory

I’m more comfortable with the latter–having been a student (and victim) of it for more than a decade–but the the process of theory building is a new concept. At least to me but also, I believe, to many. The belief that a theory is fully cooked when first conceived is not the way science developed and the idea that business management theories are singular ideas rather than processes is symptomatic of an immaturity in the field.

So here are the basics of theory building as put forward by Clay Christensen and David Sundahl:

Definition: A theory is a statement of what causes what, and why, and under what circumstances. A theory can be a contingent statement or a proven statement. That is all.

Many managers shy away from using the word “theory” because it is associated with the term theoretical which suggests impractical. But managers use theory every day. They make decisions on some basis of cause and effect, often without being specific about their reasoning.

Process: First comes observation. Second, description. Third categorization. Fourth comes analysis and a statement of what causes what and why. This analysis can be simply an observation of a pattern or a more rigorous correlation analysis.

But that’s not the end of the process. The causal statement needs to be tested by predictions whose validity is tested with further observations and confirmation or denial of the statement. If the statement is denied we need to decide if it’s an anomaly that expands the theory or whether it contradicts the theory making it less useful.

The anomaly allows a new categorization to take shape. Getting the categories right is the key to the usefulness of the theory. The discovery of anomalies can thus make a theory stronger. The discovery of anomalous phenomena is the pivotal element in the process of building an improved theory.

This iteration between prediction/confirmation/anomaly handling can go for quite some time. As anomalies are accounted for on a regular basis then they can either be exhausted or depleted enough that a robust enough categorization emerges and the predictive power is nearly complete.

Example: In my reading of Apple’s financial statements I observed that Capital Expenditures were rising dramatically after the company began to sell iPhones. The observations were made over a few years. The pattern observed showed some correlation between spending and shipments of units.

The company’s spending was then compared with a group of other technology companies. These observations suggested that spending varied according to business model and strategy and that Apple seemed to be transitioning from one type of spending (on infrastructure) to another (on manufacturing equipment.)

Then a statement was made that Apple was using capital expenditures to not only ensure supply of components but also of component manufacturing equipment. This was borne of necessity but had the side effect of creating competitive advantage as its unibody devices and Macs were unique and differentiated.

As the more data came in, by the prediction was made that capital expenditures– which are incurred before production starts and which are pre-announced on a fiscal year basis — indicate new product ramps or new product introductions.

A few anomalies were experienced when spending increased but production didn’t and vice versa. These were studied and explained by shifts in technology (mainly screens) which required “out-of-phase” investment. Additionally, the companies in the cohort also varied their spending on the basis of opportunities in the short term.

As it stands, the theory that Apple uses capital investment in tooling to manage its quality and quantity of production and that in doing so it integrates deeply into its supply chain creating competitive lock-outs is holding up. It is not sufficiently precise to forecast actual production volumes for individual product lines but the growth in the business is broadly foretold by the growth in capital expenditures.

Indeed the share price generally reflects this:

Screen Shot 2014-09-30 at 5.31.38 PM

Proposition: At a basic (micro) level, the process of theory building is something we do instinctively. We observe patterns, make statements that A causes B and carry on with the theory as an assumption. The challenge is more on a macro level. Few theories are built rigorously about the causes of success or failure of business systems. This includes understanding why large, powerful firms fail when confronted with small, weak competitors. Why, how and when industries disappear. How resources are allocated and how priorities are set. It’s as if individuals behave with far more intuitive insight than firms.

That is what must change.

Because firms are increasingly determining the prosperity and sustainability of nations and the world. We can’t afford mismanagement.

The counter-point to this quest is that large systems are intractable and business is inherently chaotic, unpredictable. It may be, but much of what we know as science today was once thought of as impossibly mysterious and unknowable. I have faith that as the physical universe, the affairs of men have laws which govern them.

Revolutionary User Interfaces, Part 2

In 2011 I wrote:

My hypothesis is that The Primary Cause for the shift of profits from Incumbents to Entrants has been the disruptive impact of a new input method.

It was a description of what I considered to be the “disruptive technology” which caused incumbents which had a “front-row seat” to the future of their industry to be completely displaced and marginalized by an entrant[1] with no discernible right to do what they did.

I illustrated what underpinned the sea change in the phone business via the slide that Steve Jobs used in the iPhone launch event:

Screen-Shot-2011-11-03-at-11-3-10.45.20-AM

 

I added the years when each input method was introduced and the  platform/ecosystems created as a result. These new ecosystems were the primary cause for dramatic industry-sized shifts in profits.

Not coincidentally, during the 2014 Apple Watch launch, the presentation began[2] with a re-telling of the “mouse, click wheel and Multi-Touch” story.

Screen Shot 2014-09-10 at 10.07.55 AM

Seven years later, the difference is that there is a new object added to the story. It answers the question that has been on my mind since that first post on revolutionary user interfaces was written: what will come next.

Now that we have an answer, the next step is to understand the new platform, its ecosystem; which industry will be affected and which incumbents will be displaced and to what degree will value be created beyond that which will be displaced.

Piece of cake.

Notes:
  1. later more than one []
  2. Begins one hour into the 2 hour downloadable video []

Going where the money is

The bank robber Willie Sutton did not say, when asked why he robbed banks, “because that’s where the money is.” He did agree with the idea however saying “Go where the money is…and go there often”.

Regardless of it being apocryphal, this idea came to be called Sutton’s Law and is often taught to medical students. It’s similar to the notion of Occam’s Razor: when an obvious or simple answer competes with an obscure or complicated answer, pick the obvious one first.

These are sound analytical rules of thumb. When thinking about what products and services could arise in the immediate future, those most obvious and with fewest assumptions should be put forward first. The what part is relatively easy. The tough question is more about when will they emerge?

We now know that Apple will announce new products on September 9th[1]. This gives us an idea of when something will happen, answering the tougher question. It leaves the simpler question of what will emerge.

I put forward my predictions as follows:

  • Regarding iPhone, a tweet on product mix and pricing.
  • Regarding an “iWatch”, an answer to a question from Eric Jackson.
  • Regarding the potential for wearables, a post on the subject.

One more item has surfaced on the potential of payments processing which I want to address now.

Handling payments, to me, is a perfectly plausible activity for Apple mostly because the company has made quite a few comments on the value of their “customers with credit cards” and the effort that went into Touch ID (which seems to be extravagant relative to the value of rapid unlocking).

But one word of caution: if Apple does enable payments it’s important to realize that being a (payment) bit pipe is not a particularly profitable business. It will undoubtedly bind value to the iOS devices which make it possible, but I don’t think there will be a direct capture of profit from the transactions themselves.

Notes:
  1. I’ll be there and will report via Twitter and a special session of The Critical Path podcast []

Apparel is next

If software can be injected into an industry’s product it will bend to the will of the software writers.

This theory expands on Marc Andreessen’s observation that “software is eating the world”. The evidence is that software, coupled with microprocessors, sensors, batteries and networking becomes applicable to an increasingly larger set of problems to be solved[1]. Software has “eaten” large portions of entertainment (e.g. Pixar, iTunes, video games), telecommunications (iPhone, Android, Messaging), various professions including journalism, management and law, and is entering transportation, energy and health care and poised over banking, finance and government.

As entry happens, asymmetries are enabled and disruption follows. This is the bending to the will of the writers–who tend not to be incumbents. The incumbents can’t embrace the changes in business models enabled by software without destroying their core businesses and thus, invariably, they disappear.

The pattern is easily observed but the speed and timing of it is difficult to predict and hence investment success is not certain.[2] There are many entrants who try and few succeed and there are many incumbents who will survive longer than a prophet can stay hungry.

Nevertheless, this process of software-induced turnover in wealth–and, incidentally, vast, additional wealth creation–is inevitable.

But can we predict anything other than timing? For example, can we predict the next industry to succumb to this force?

Notes:
  1. Or, put another way, is eligible to be hired to perform an increasingly large set of jobs []
  2. Which, ironically, means that the jobs of venture capitalists are still safe. At least until the theory develops to the point where it can predict with more accuracy winners and losers. []

Beleaguered

Amazon’s recent disputes with publishers (Hachette and Disney) shows a degree of market power that is closer to monopsony than to monopoly but this power is nevertheless real. It may not not be something that requires intervention, regulation or even scrutiny but market power is evident in both how companies operate and in how they are valued.

If you look at the following graph, it’s easy to spot those with “monopoly” power. The graph shows a short history of revenues/operating income and P/E ratios. Modest or no growth in earnings coupled with extraordinary high P/E ratios indicate that the market understands the business is not threatened by competition.

Screen Shot 2014-08-14 at 8-14-1.40.47 PM

On Capital Allocation

One of the paradoxes of the “post-industrial” era is the aversion to application of capital to growth opportunities. Generally speaking, capital has become trapped in bank accounts as opposed to equipment which could be used to produce value. This aversion is rooted in many dysfunctions, chief among them being the misunderstanding of the purpose of the firm.

But there are exceptions. Illustrated below are the patterns of spending in property plant and equipment (capital expenditures) by companies that still recognize that there are opportunities to be obtained by investment in the means of production.

Screen Shot 2014-08-13 at 11.36.03 AM

Competing effectively against your most potent competitor

New market disruptions take root in non-consuming contexts. For instance, mobile phone photography began not because early phone cameras were good. They weren’t good at all but good enough when a camera was not within reach. The quality was poor but the photo taken would not have otherwise been taken, making a lousy photo better than no photo.

The result is that the total number of photos taken this year will be ten times higher than the total number of photos taken before the advent of mobile phone cameras.[1]

This rush to use the phone as a camera has meant that phone makers are keen to improve their product (so as to compete effectively with it against each other) and as a consequence they overtake the incumbent camera makers in quality as well as quantity.

The same phenomenon was experienced by fixed component “Hi-Fi” audio products. The quality of mobile music was poor but it was convenient and convenience translated into consumption and consumption translated into quality improvement and eventually the evaporation of usage of the traditional category.

Now consider how ad dollars are getting spent. The following chart shows the eMarketer forecast for ad spending mix across different media in the US.

Screen Shot 2014-07-02 at 7-2-8.12.01 PM

It would appear that the “Mobile” media is competing effectively against the other media types, especially the non-Mobile digital (i.e. PC-based experiences).

However, if we look at the absolute spending forecast the picture shows that Mobile is responsible for most of the growth in the overall spending.

Notes:
  1. The total number of photos taken in 2014 is likely to be around 880 billion. Prior to 2000 the total number of photos ever taken is estimated at 85 billion. []

The Disruption FAQ

Q1. What is disruption?

Disruption happens when the strong are defeated by the weak. More precisely it’s when those with unconstrained access to resources have them taken away by those with minimal or no resources. It’s a phenomenon that is in contrast to sustaining competition where the strong get stronger.

Q2. How can disruption happen? Don’t the strong always have an advantage over the weak?

The strong can be defeated when the fight is unfair. More precisely, “strength” is only a perception based on convention or historic precedent. The entrant may be weak in resources but may be strong in a way that is not seen as conventionally useful or valuable such as agility or a willingness to learn.

Q3. What makes a fight unfair?

A fight is unfair if the opponents fight according to different rules. This is also called asymmetric competition. Asymmetry is an important concept in game theory, economics and military science.

Q4: Is there a way to know disruption is about to happen?

Disruption theory is an attempt to reliably identify winning challenges. It includes a method of analysis of “the setting” of the fight and “the weighing” of the fighters. In other words, it measures whether the challenger is sufficiently asymmetric and whether the incumbent is flexible enough in their likely response. If there is insufficient asymmetry the theory would suggest the challenger will lose, and vice versa.

Q5: How often does it happen?

In some industries it happens quickly and in some industries slowly and in some never at all. Determining the cause of the rate of disruption is an important research topic. We can hypothesize that industries which show frequent disruption also show a high degree of wealth creation. The inverse is also true: industries that don’t get disrupted don’t create much wealth.

Q6. How do incumbents react to asymmetric challenges?

This quote (mistakenly attributed to Gandhi) describes it best: “First they ignore you, then they laugh at you, then they fight you, then you win.” The main test of asymmetry is to ask whether a challenger’s entry is ignored (or welcomed) by the incumbents. Most asymmetric challenges are not taken seriously because they initially benefit the incumbent. The side-effect is that it lulls them into a sense of security resulting in a lack of response. Challengers have the child-like advantage of rapid growth and learning while incumbents are encumbered by their size and lack of flexibility.

Q7: Why don’t challengers respond in kind?

Mainly because they don’t feel that they need to. The newcomer is either not seen as a threat or welcomed because the customers they obtain that are not seen as valuable. In some cases the business model of the entrant is contrary to that of the incumbent. In other words, the challenger makes money in a way that would cause the incumbent to lose money. Often the challenger is actually invisible because she misdirects attention. They may be seen as competing in one market with a certain set of competitors but their effect is in another market which never had to deal with a challenger.

Q8: If there are different ways disruption can be observed, doesn’t that mean there are different types of disruptions?

Yes. When a competitor challenges with a cheaper product that seems to perform poorly and has low margins and the incumbent accepts the entry because it allows them to concentrate on more profitable customers then this is called a “low-end disruption”. Here the asymmetry is in cost structure. The entrant has lower costs and accepts lower profit margins and may “make money” in new or different ways.

When a competitor misdirects attention by selling a product that draws usage from existing customers and adds non-consuming new customers because it enables new uses, then the incumbent feels no pain from the entry because they don’t sense a reduction in customers. We call this a “new market disruption“. The challenger gains a foothold and grows/evolves, eventually capturing customers exclusively. Here the asymmetry is in the basis of competition and measurement of “performance”. The new product does not actually do the same thing as the incumbent product or does a subset of valuable tasks poorly while excelling at menial tasks. The entrant may be highly profitable but they are not taking profits away from incumbents because they “grow the pie”, capturing value by fulfilling unmet needs.

Disruption can also happen to professions and institutions when less skilled individuals are enabled to perform complex jobs or when professionals can establish good enough services that used to take institutional support. This is called professional services disruption.

Technological change is often (but not always) the core enabler in creating disruptions. The analogy is that technology is a weapon that allows asymmetric combat but the combatant is the disruptor, not the weapon[1].

Q9: It sounds like the trick to spotting disruption is in perceiving when the fight becomes “unfair” and an outsider advantage is gained. Does the theory make this easy?

It makes it possible but not easy. Understanding when a basis of competition changes and where competition is shifting is still very difficult. It is notoriously difficult to sense when it’s happening to you because you are working toward a strategy with assumptions that have been tested and proven to be correct. In other words, you, your colleagues, your competitors and everyone you’ve ever met knows the rules of the fight. Insider status makes you an expert, your knowledge is far beyond a lay person’s and you have a track record of winning. Hubris and pride makes it difficult to accept a challenge from an ignorant outsider.

Furthermore, an outside analyst who is not suffering from these psychological weaknesses may not have the means to measure change because they don’t have access to market metrics. So you can’t typically hire a consultant to help you spot it. This is where the theory helps. If you are a practitioner then you can use the theory as a lens to see the patterns in the operating data.

The proper application of the theory requires domain knowledge or deep reading of weak signals. It’s best employed by incumbent operating managers analyzing their own industry. It’s a tool that can be used to overcome the perception of incumbent invulnerability. It is not a tool that can be used by armchair generals. They can pick up the lens but, not having any data to look at, they have have no patterns to recognize. (This last point is disputed. See comment.)

By the way, entrants don’t benefit much from it either because they act disruptively by instinct. They enjoy the freedom of having nothing to lose.

Q10: How long does this take? Isn’t a shift in competition natural over a few decades?

The speed of disruption is changing rapidly. It used to take decades but now it takes years and in some software industries it could be happening in less than one year. When it used to take decades it did not matter much because the “victims” of a disruption usually could spend a career in the firm being disrupted and would not have to adjust their behavior or assumptions. The consequences would have been felt by future generations. The rate of disruption today is so rapid that many careers and lives and families are having to deal with the consequences, sometimes more than once. Estimating the acceleration and scope of disruptive change is a great research topic.

Q11: Is the theory complete? Can’t we just write an app for it?

It’s not complete. It cannot be encoded as a deterministic algorithm. It’s not even likely that expert systems or neural networks or machine learning can help. This is because perception of change in competition is a skill informed as much by intuition as by data and rules. If the theory is developed further, through a process of theory building, then it might become an app.

Q12: If the theory is not complete, then isn’t it useless?

A theory is not useless if it’s imprecise or difficult to use. Data that would make conclusions precise is often missing or unavailable. The theory relies on weak signals and explains what used to be unexplainable. Many sciences developed from empirical analysis similar to where we are with disruption theory. Theories benefit from development.

Q13: Is the theory being developed further?

Yes. There is a process for theory building which has been ongoing for over a decade where many researchers, students of disruption and practitioners are contributing.

Q14: Can the theory be applied outside of business competition?

Yes, institutions can be disrupted as can individuals. Possibly even economies and states. This is because a growing technological base and communications enable asymmetries of ever-greater scope and speed. The research needed to establish how this happens is under way. However it’s important to know the limits of the phenomenon. Incumbents are getting wiser as they wield the theory and some “settings” show resistance to change and thus prohibit technological cores to be utilized disruptively. The study of these anomalies is essential to the process of theory building.

Q15: Is disruption a force of nature? Has it always existed? Will it never end?

Oral traditions suggest that people have been aware of disruptive forces for all of history. We feel it in the school playground and in many personal relationships. It’s in the Bible and the classics that predate it. It’s visible in all cultures. Disruption theory as applied to business and government is only an extension of this causal individual behavior into complex systems. It’s possible that awareness of it might cause the behavior to change, or, put another way, that if you observe and understand the phenomenon, that knowledge could cause it to stop happening. But the systems involved are vast and learning curves are long. Enlightenment may take a few lifespans.

Update:

Here are questions which have been added based on reader feedback. Please feel free to suggest others in the comments.

Q16: Isn’t Disruption just evolution of business models?

Business models evolve, but if the evolution sustains an existing incumbent then it’s not a disruption. If an evolution is  adopted by an entrant who then proceeds to strip the assets and profits from an incumbent then it’s a disruptive change. The key question is why do some changes get adopted and not others.

Q18: What have been the major improvements in the theory since its introduction in 1997’s The Innovator’s Dilemma?

From Clay Christensen’s work alone: New Market Disruptions, Professional Disruption, Value Chain Evolution Theory, Jobs to be Done Theory. The Innovator’s Solution including how to respond with autonomous self-disruption, the role of acquisitions, the study of Healthcare and Education as targets of (and anomalies to) disruptive change. The process of theory building. The Capitalist’s Dilemma and how economies create incentives and disincentives to disruptive innovations.  Many other contributions from researchers too numerous to cite here.

Q19: What industries cannot be disrupted

Industries which experience no disruptions are “anomalous” in that the theory suggests that technological progress forces change and technologies have proliferated in almost all industries..  Industries or institutions which have remained largely undisrupted include Energy, Education, Government, Healthcare, Airlines and Hotels. The study of these anomalies continues and explanations identify conditions that prevent growth. Dependencies on regulation, infrastructure, and absence of technological core enablers caused some of the atrophy to date. However signals of change are appearing in all these industries and paths to disruption are clearly possible.

Q20: How is it possible that companies which were claimed to be disrupted are still around and some which are claimed to be disruptors have vanished?

Being disrupted does not mean ceasing to exist. Being disrupted is a loss of a specific encounter, but not necessarily a terminal loss. Being a disruptor is a win in a specific encounter and not a guarantee of immortality. Indeed many disruptors come to be disrupted and the theory suggests sustainable growth requires self-disruption. Also a disrupted company can be re-configured post-disruption into another entity or can muddle along with limited value indefinitely. Sometimes they can rise up and become disruptors again.

Notes:
  1. See David vs. Goliath: using a projectile allowed David to overcome Goliath but only when wielded skillfully []

Ten years ago: Clayton Christensen on Capturing the Upside

You can hear this as an MP3.

[It's important to understand just how much the theory has evolved in the last 10 years. Much more perhaps than in its first eight.]

Doug Kaye: Hello, and welcome to IT Conversations, a series of interviews recording and transcripts on the hot topics of information technology. I am your host, Doug Kaye, and in today’s program, I am pleased to bring you this special presentation from the Open Source Business Conference held in San Francisco on March 16 and 17, 2004.

Mike Dutton: My name is Mike Dutton, and it is my pleasure to introduce to you today Clayton Christensen. Professor Christensen hardly needs an introduction. His first bestseller, “The Innovator’s Dilemma,” has sold over half a million copies and has added the terms “disruptive innovation” to our corporate lexicon. His sequel — and you have to have a sequel to be a management guru — is entitled “The Innovator’s Solution” and is currently Business Week’s bestseller’s list. Professor Christensen began his career at the Boston Consulting Group and served as a White House fellow in the Reagan administration. In 1984, he cofounded and served as chairman of Ceramics Process Systems Cooperation. Then, as he was approaching his 40th birthday, he took the logical step of quitting his job and going back to school, where he earned a doctorate in Business Administration from Harvard Business School. So, today he is a professor of Business Administration at Harvard Business School where teaches and researches technology commercialization innovation. Professor Christensen is also a practicing entrepreneur. In 2000 he founded Innosight, a consulting firm focused on helping firms set their innovative strategies. And according to a recent article in Newsweek, “Innosight’s phones ring off the hook, and the firm cannot handle all the demand,” very similar to all the startups in open source here today. So, please join me in welcoming Clayton Christensen.

Clayton Christensen: Thank you, Mike! I’m 6 feet 8, so if it’s okay, I’ll just…the mic picks up okay. I’m sure delighted to be with you, especially because there is blizzard in Boston today; my kids have to shovel the snow!

As Mike mentioned, I came in to academia late in life, and the first chunk of research that I was engaged in was trying to understand what it is that could kill a successful, well — run company. And those of you who are familiar with it, probably know that the odd conclusion that I got of that was that it was actually good management that kills these companies. And subsequent then to the publishing of the book that summarized that work, “The Innovator’s Dilemma,” I’ve been trying to understand the flip side of that, which is if I want to start a new business that has the potential to kill a successful, well — run competitor, how would I do it? And that’s what we tried summarize in the book, “The Innovator’s solution.” It’s really quite a different book than the “Dilemma” was, because the “Dilemma” built a theory of what is it that caused these companies to fail. And then in the writing of this solution, I’ll just give you analogy for where we came out on how to successfully start new growth businesses.

I remember when I first got out of business school and had my first job. I was taught the methods of total quality management as they existed in the 1970’s, and we had this tool that was called a “statistical process control chart.” (Do they still teach that around here?) Basically you made a piece, you measured the critical performance parameter and you plotted it on this chart, and there was a target parameter that you were always trying to make the piece to hit, but you had this pesky scatter around that target. And I remember being taught at the time that the reason for the scatter is that there is just intrinsic variability and unpredictability in manufacturing processes.

So, the methods that were taught about manufacturing quality control in the ‘70’s were all oriented to helping you figure out how to deal with that randomness. And then the quality movement came of age, and what they taught us is, “No, there’s not randomness in manufacturing processes.” Every time you got a result that was bad, it actually had a cause, but it just appeared to be random because you didn’t know what caused it. And so the quality movement then gave us tools to understand what are all the different variables that can affect the consistency of output in a manufacturing operation. And once we could understand what those variables were and then develop methods to control them, manufacturing became not a random process, but something that was highly predictable and controllable.

▶ Horace Dediu: “Transformation of Business and Society through Technology”

My talk on the future of things from Censhare’s FutureDay 2014 in Munich.