Executive Summary
What if innovation were not a creative gift that some organisations have and others lack, but a measurable engineering discipline that any organisation can learn to manage? That is the central argument of Professor David Cropley of the University of Adelaide, one of the world’s leading researchers in engineering innovation. In this episode of Arkaro Insights, David explains why most organisations are systematically failing at innovation, where exactly the failure occurs, and what leaders can do about it.
Core insights:
- Innovation is not a light bulb moment or a spark of creative genius, but a structured, measurable process with identifiable stages, psychological dimensions, and diagnostic tools.
- The Innovation Phase Model maps seven stages of innovation against six psychosocial dimensions, producing a 42-cell diagnostic matrix that reveals precisely where an organisation’s innovation capability breaks down.
- Most organisations are competent at the implementation phase, exploiting what they already know, but consistently fail at the front end: problem identification, idea generation, and idea evaluation.
- The explore versus exploit tension is structural, not accidental. Most promotion and reward systems are calibrated to exploitation, making exploration the rational individual choice to avoid.
- AI is being misapplied in most organisations because it is being treated as a product innovation when it is fundamentally a process innovation, a category error that will produce the familiar failure modes of time overrun, cost overrun, and underperformance.
- David has demonstrated mathematically that large language models are capped at approximately average human creativity (0.25 on a normalised scale), a useful capability but not a substitute for genuine inventive thinking.
- Building a sustainable innovation culture requires three things: making innovation pervasive rather than confined to an R&D team, investing real time and money in it, and maintaining a proportion of genuinely disruptive thinking alongside incremental improvement.
- The Smith Corona story is the clearest available illustration of what happens when a world-class organisation innovates only incrementally in the face of a disruptive technology shift.
Innovation Is an Engineering Discipline, Not a Creative Spark
The most common misconception about innovation is that it is essentially creative, mysterious, and resistant to management. Either your organisation has it or it does not. Either the right people have the spark or they do not.
David Cropley has spent 25 years dismantling that assumption.
His starting point is straightforward. If we define creativity rigorously, as researchers in the field do, it is possible to assess whether an organisation’s conditions support or inhibit creative output at each stage of the innovation process. And if we understand the mathematics of how a technology such as a large language model works, we can calculate with precision what level of creative output it can and cannot produce.
This is the engineering mindset applied to innovation: not mysticism but measurement, not inspiration but diagnosis.
The practical implication is significant. If innovation failure is measurable, it is manageable. If the failure points are identifiable, they are addressable. The question shifts from “are we an innovative company?” to “at which stage of the innovation process are we losing capability, and why?”
The Innovation Phase Model: A 42-Cell Diagnostic
The theoretical foundation of David’s work is the Innovation Phase Model, developed with colleagues including his father, Professor Arthur Cropley, a longstanding creativity researcher at the University of Hamburg.
The model identifies seven stages through which individuals and organisations must pass to innovate successfully: recognition that a problem exists, preparation, ideation, evaluation, elaboration, implementation, and validation. Across those seven stages, the model maps six psychosocial dimensions, drawing on decades of creativity research to specify what conditions, individual properties, and organisational climate factors are required at each stage to maximise the probability of success.
The result is a seven-by-six matrix of 42 cells, each describing a specific combination of innovation stage and psychosocial requirement. From this matrix, David and his colleagues developed a 168-question survey instrument, with four questions per cell, all framed around the stem “in this organisation…” The survey is designed to be administered to teams or whole organisations, typically with samples of 50 to 200 people, and the aggregated responses are mapped against the theoretical ideal encoded in the matrix.
What emerges is not a generic innovation health score but a granular diagnostic that identifies exactly which cells, which combinations of stage and dimension, represent the organisation’s strengths and which represent its roadblocks.
Managers who have received the results consistently report the same reaction: confirmation of what they already sensed. The value of the instrument is not in surprising people with unknown problems but in providing the evidence base to act on problems that were already suspected. Suspicion becomes diagnosis. Diagnosis enables intervention.
Where Innovation Actually Fails: The Front End Problem
Across the dozen or so organisations David has surveyed, a clear pattern has emerged. Most organisations are reasonably competent at implementation: the stage at which a defined solution is built, deployed, and delivered. They have evolved processes, structures, and skills for executing known tasks. This is exploitation, and the reward systems of most organisations are well calibrated to it.
Where organisations consistently struggle is the front end: problem identification, ideation, and evaluation. Generating genuinely new ideas in response to poorly defined or novel problems is where the roadblocks accumulate. And this is not a coincidence. It reflects something structural.
As David notes, structure determines behaviour. If the incentive system rewards exploitation, people will exploit. If promotion depends on short-term, high-certainty results, rational individuals will focus on short-term, high-certainty activities. The executives who talk about innovation and the systems that reward exploitation are often the same people operating the same organisation. The tension is baked in.
This is why innovation cannot be fixed simply by exhortation or by hiring creative people. The conditions that enable the front end of innovation, psychological safety, tolerance for divergent thinking, time and space for exploration, support for risk-taking, must be actively created and maintained. Without them, even talented people will default to exploitation because exploitation is what the organisation actually rewards.
The AI Misapplication: A Category Error with Predictable Consequences
Much of the current enthusiasm around AI in the workplace illustrates exactly the failure mode that David’s framework predicts. Organisations are treating AI adoption as a product innovation: a new tool to be bolted onto existing processes. The planning is product-style planning. The governance is product-style governance. The measurement is product-style measurement.
But AI adoption is a process innovation. It changes how work gets done, how decisions are made, how value is created and captured. Treating it as the wrong category of innovation means organisations are misdesigning the entire programme from the outset.
The consequences are predictable from the innovation literature: projects that overrun their timelines, cost more than planned, and fail to deliver what was promised. Not because the technology is inadequate, but because the organisations have skipped the front end entirely, forgoing problem identification and problem framing in the rush to deploy something new and visible.
David’s advice is to slow down. Not because AI is unimportant, but because the history of major technology investments demonstrates consistently that those who define the problem carefully, design the solution rigorously, and manage implementation systematically outperform those who deploy quickly without that foundation. The organisations that will extract genuine value from AI are those that apply an engineering discipline to its adoption, treating it as a process innovation from the start.
The Mathematical Ceiling on AI Creativity
One of the most distinctive contributions David brings to the AI debate is mathematical rather than philosophical. He has published a paper demonstrating that large language models have a calculable upper bound on their creative output.
The argument is elegant. We have well-established, agreed definitions of creativity in the psychological literature. We also have a clear mathematical description of how large language models work, essentially as weighted probability functions over large corpora of existing human output. Combining those two things produces a calculable answer.
On a normalised scale from zero (no creativity) to one (maximum creativity), large language models sit at approximately 0.25. That corresponds roughly to average human creativity. The model is not useless at creative tasks. Average creativity is a meaningful capability, in the same way that a car capable of 20 miles per hour is a useful vehicle. But it is not a ceiling that further data or incremental model improvements can push through. Getting beyond it requires a different approach to the technology.
The implication for innovation leaders is practical. Large language models are useful tools in the innovation process, particularly at stages where generating a volume of candidate ideas quickly has value. But they cannot replace the genuinely inventive thinking required at the front end of a serious innovation challenge. Using them as if they can is another category error, with the same predictable consequences.
The Smith Corona Warning: Incremental Innovation Is Not Enough
David returns repeatedly to the Smith Corona story as the cleanest available illustration of what happens when innovation culture is healthy but incomplete.
Smith Corona was not a complacent company. From its founding in the late nineteenth century, it consistently innovated: mechanical to electric typewriters, the golf-ball print head, self-correcting mechanisms. It was excellent at its craft. But its innovation was entirely incremental, always the best typewriter rather than a question of whether the typewriter was still the right product category.
When IBM introduced the personal computer in 1981, the category question could no longer be deferred. Smith Corona’s response was to make better typewriters. It went bankrupt. Reconstituted, it made the same mistake again and went bankrupt again. The vestiges of the business survive today only in label-making tape, a peripheral technology that survived the disruption.
The lesson David draws is not that disruptive innovation always wins but that any organisation requires a proportion of genuinely disruptive thinking alongside its incremental improvement programmes. Nobody knows when the next Smith Corona moment will arrive. The organisations that have embedded disruptive innovation capability are ready for it when it does. Those that have only incremental innovation capability are not.
For leaders in agriculture, food, and chemicals, the relevance is direct. Regulatory change, new technologies, shifting consumer expectations, and geopolitical disruption are not theoretical risks. The organisations that treat innovation as an engineering discipline, maintaining both incremental and disruptive capability, are building resilience. Those that treat it as a light bulb moment, hoping the spark arrives when needed, are taking a risk they may not survive.
Three Things a New Leader Should Do First
Asked what a newly appointed CEO or divisional head should prioritise to build genuine innovation capability, David offered three answers.
First, make innovation pervasive. Confining it to an R&D team or a skunk works, however talented, will not produce the cultural shift required. Innovation that depends on a small team is fragile and limited. Innovation embedded across the organisation is robust and scalable.
Second, be prepared to invest. Innovation capability does not emerge for free. It requires time, money, tolerance for failure, and the deliberate redesign of systems that currently reward only exploitation. Leaders who want innovation without investment are asking for something that does not exist.
Third, balance incremental and disruptive. The day-to-day pressure is always towards incremental improvement, and incremental improvement is genuinely valuable. But some proportion of the innovation effort must address genuinely new directions, new categories, new business models. Not because disruption is always the answer, but because the absence of disruptive capability leaves the organisation defenceless when the environment changes fundamentally, as it always eventually does.
About the Guest
Professor David Cropley is a Professor of Engineering Innovation at the University of Adelaide, Australia, and one of the world’s leading researchers at the intersection of engineering, creativity, and innovation. His work applies rigorous psychological and mathematical frameworks to the practical challenge of building innovation capability in organisations. He is co-author, with Professor Arthur Cropley, of The Psychology of Innovation in Organizations, published by Cambridge University Press.
David is available for engagements on engineering innovation diagnostics and organisational innovation capability. Contact him via LinkedIn or through the University of Adelaide.
LinkedIn: linkedin.com/in/davidcropley
Mark Blackwell is the founder of Arkaro, a B2B consultancy specialising in change management, innovation, and commercial excellence for the agriculture, food, and chemicals industries. Arkaro’s collaborative “do it with you” approach works closely with clients through four steps, Understand, Co-create, Enable, Sustain, to leave behind sustainable, value-generating solutions, not just a slide deck.
Website: arkaro.com | LinkedIn: linkedin.com/in/markrblackwell
Related Reading from Arkaro
On the Front End of Innovation: Problem Finding and Idea Generation
Why Teams Pick the Wrong Ideas — and What Leaders Can Do About It | Dr Roni Reiter-Palmon — David Cropley identifies the front end as where most organisations fail. Roni Reiter-Palmon’s research explains precisely why: 45% of teams select suboptimal ideas because they have never properly agreed on the problem. Problem construction is the step that precedes everything else, and most organisations skip it entirely.
70% of People Are Wrong About Creativity | Yale Researcher Dr Zorana Ivcevic Pringle — Where David argues that innovation is an engineering discipline, Zorana Ivcevic Pringle argues that creativity is a learnable decision-making process, not a talent. Her finding that the most creative teams spend 53% of their time framing the problem reinforces David’s front-end diagnosis from a different research tradition.
The Innovation Strategy Imperative: Why “Quick Win” Projects Often Fail — David’s exploit-versus-explore tension plays out in practice as the quick win trap. When organisations lack a clear innovation strategy aligned to business objectives, they default to low-ambition, short-horizon projects that deliver poor returns and crowd out genuine innovation. This article explains the mechanism and how to break the cycle.
On Why Innovation Fails in Organisations: Culture, Governance, and Structure
Why Product Launches Fail: It’s Not What You Think — David’s 80% new product failure rate is not a coincidence. This article examines the root causes, including poor cross-functional collaboration, unclear innovation strategy, weak value propositions, and flawed business models, all of which originate long before launch. The diagnosis maps directly onto David’s innovation phase model.
Building the Foundation: How Collaborative Culture Impacts Innovation Success — David’s 42-cell diagnostic consistently reveals cultural and climate factors as key failure points, particularly at the front end. This article explains why collaborative culture is not a soft dimension of innovation but the foundation on which technical capability depends.
Governance: The Critical Link Between Innovation and Commercial Success — If David’s diagnostic identifies where roadblocks exist, governance is the mechanism for addressing them. This article identifies the five governance failures that undermine innovation programmes and explains what effective governance looks like in agriculture, food, and chemicals organisations.
On AI, Disruption, and What Comes Next
Beyond the Hype: What Creativity Research Tells Us About AI and Innovation | Dr Todd Lubart — The most direct companion piece to this episode. Where David demonstrates mathematically that large language models cap at average human creativity, Todd Lubart explores from a psychological research perspective what human-AI collaboration in creative work actually looks like. Together they form the most rigorous treatment of AI and creativity in the Arkaro Insights back catalogue.
Constraints, Playfulness and Ethics: 3 Lessons for Leading in the AI Age | Dr Vlad Glaveanu — David’s argument that AI is being misapplied as a product innovation rather than a process innovation connects directly to Vlad Glaveanu’s three leadership lessons for the AI age. Glaveanu’s argument that constraints enable creativity and that ethics must guide what we build provides the leadership frame for David’s engineering diagnosis.
Why Smart Companies Miss Disruption: The 3 Ghosts Blocking Innovation | Scott Anthony — Smith Corona is David’s cautionary tale. Scott Anthony’s three organisational ghosts, the past traumas, present blind spots, and identity fears that stop established companies responding to disruption, explain the psychological mechanism behind exactly that kind of failure. Read alongside David’s episode, this article completes the picture.
AI and Jobs to Be Done: Where Human Judgement Trumps Technology — David shows mathematically where AI creativity reaches its ceiling. Scott Burleson’s work on AI and Jobs to Be Done identifies practically where human judgement remains irreplaceable, specifically in abductive reasoning and the prioritisation of customer needs. The two arguments reinforce each other.
External Resources
University of Adelaide — Professor David Cropley: Connect via linkedin.com/in/davidcropley or search “David Cropley University of Adelaide” for publications and research profile.
The Psychology of Innovation in Organizations — David Cropley and Arthur Cropley, Cambridge University Press. Available via Amazon and Cambridge University Press directly.
Arkaro Insights Newsletter: Subscribe on LinkedIn for regular perspectives on change management, innovation, and commercial excellence in agriculture, food, and chemicals.
