In an earlier article, I described the types of organization and personal qualities that promote innovation. The next two articles will explore the reasons some structures and qualities yield innovation and some do not. Here we’ll examine what the management literature and the psychological and social sciences have to say. Since much has been written on these subjects, we’ll cover a sample to convey an idea of what’s available.

A 2014 Harvard Business Review article on leadership for innovation, Collective Genius,1 was co-written by a Professor of Business Administration, two experienced high-level managers and an MIT doctoral candidate. They examined what exceptional innovation leaders think about and what they do.

Although the executives that the authors examined were in different fields, they thought about leadership in a similar way. For example, they agreed that when the solution to a problem is known and straightforward, the function of leadership is to set directions for the group. If the problem requires an original response, however, leaders have to rethink their roles. They have to find a way to manage their teams to draw on the key strengths of each individual and creatively combine those strengths.

The challenge is how to set the stage for that to happen. Especially in groups of talented and assertive individuals, managers must create an environment in which the participants are willing to express their ideas and at the same time actively listen to others.

The function of successful innovation leaders is to solve problems by integrating ideas — combining options A, B, and C, even if at one time they seemed mutually exclusive — to create an optimal choice.

Innovation Management at Google

Bill Coughran, an innovation leader described in Collective Genius, was brought to Google as senior vice president of engineering to expand the company’s data storage system. During early discussions, self-organizing groups of engineers formed around two promising alternatives. Coughran allowed these groups to pursue their preferred options, but monitored the groups closely so that each was moving in a productive direction. The result of this two-tiered process was that one solution — a so-called “sustaining innovation” — was used as a short-term fix. This allowed time for the other group to develop a new storage system that emerged as a “disruptive innovation.”

As explained in the earlier article, sustaining innovations improve the performance of established products, while disruptive innovations create something quite different from what existed before.

The Tasks of Creative Leaders

The Harvard Business Review study concluded that every creative leader has to navigate a series of paradoxes in order to ensure that he or she has a successful team. These paradoxes, as listed by the study, include:

· Affirming the individual AND the group;

· Supporting AND confronting;

· Fostering experimentation and learning AND performance;

· Promoting improvisation AND structure;

· Showing patience AND urgency;

· Encouraging bottom-up initiative AND intervening top-down.

One of the many useful articles in Organizational Reality/Reports from the Firing Line2 is about organizational politics. It begins with a quotation from Machiavelli’s The Prince: "The innovator makes enemies of all those who prospered under the old order, and only lukewarm support is forthcoming from those who would prosper under the new."

The story of Chuck House, an engineer at Hewlett-Packard, demonstrates how a maverick within a large organization can become a successful innovator. House helped develop an improved video monitor for use in an airport control tower. The HP monitor lost out to a competitor, however, because its resolution was not high enough, even though its smaller size, faster speed, energy efficiency, and brighter picture were significant breakthroughs.

House was undeterred and took an unauthorized trip to conduct his own market research. By personally meeting with potential customers, he was able to identify a variety of applications for the new monitor. Because of that information, senior management allowed House to continue work on the project. However, when he presented his efforts at an annual division review, staff from the marketing department objected. They said that their research had identified customers for only 32 monitors. House countered by explaining that marketing had contacted only their traditional base of oscilloscope users rather than finding customers who might have new applications. David Packard, who was at the meeting, was not convinced and told House to discard the project from the laboratory.

House chose to interpret that to mean the project would no longer be in the lab, but would be in production instead. His boss covered for him and allowed him to go forward with development. Although such a project normally would have taken two years, House and his team were able to complete it in one. At the next annual review, David Packard saw the results and decided to support the project. The monitor was eventually used for the NASA moon mission, one of the first artificial heart transplants, and as part of an award-winning special effects system.

Three lessons from this article are that:

1. Developing an innovative product also requires innovative approaches to administration and marketing.

2. While innovations of a sustaining character can usually find support within an organization, disruptive innovations will be resisted, because they will be perceived as a threat to existing power relationships.

3. “Within organizations, the political strategy of ‘asking for forgiveness’ is limited to only the initial phases of a product innovation… For the adoption and diffusion of a new product, the innovator must ‘seek and secure permission’ of the organization."

The Final Report from the NSF [National Science Foundation] Innovation and Discovery Workshop: The Scientific Basis of Individual and Team Innovation and Discovery3 explores relationships between psychology and engineering. One of the workshop’s conclusions was that creativity and innovation are not the same; innovation is a subset of creativity. Innovation means creating a new idea, but also figuring out how to implement it. The report also confirms the importance of horizontal collaboration rather than top-down hierarchy.

Group Dynamics

The question as to whether innovation is created by groups or by individuals has been much debated. In his 2012 New Yorker article,4 Jonah Lehrer discusses the concept of brainstorming, which was the idea of advertising executive Alex Osborn. The concept was first published in 1948 and still has its advocates. The notion is that creativity comes from a group of people free associating ideas without criticizing each other and then evaluating the ideas afterwards. However, Lehrer describes a series of experiments performed by the psychologist Charlan Nemeth in 2003 that indicated creative ideas more often come from groups that encourage, rather than limit, internal debate and discussion.

Pluses and minuses exist to using groups as incubators of innovation. On the one hand, it’s a way of exposing people to divergent ideas and enabling them to benefit from hearing others’ responses. On the other hand, there are social-psychological pressures within groups to create and conform to a common outcome that buries difference and dissent.

Some of the negative aspects of group behavior were experimentally explored by the pioneering social psychologist Leon Festinger. The author of a recent review of Festinger’s work,5 said one of his important observations was that people “do not like inconsistency. It upsets us and it drives us to action to reduce our inconsistency.” This tendency was illustrated when Festinger and two coworkers studied a group who expected the world to end on Dec. 21, 1954. The researchers spent time with the group both before and after that date.

The result was that rather than dispersing after the December date, the members sought ways to avoid the obvious: that their beliefs were wrong. They drew together more closely and tried to attract new people to their group in order to prevent it from falling apart. Festinger coined the term “cognitive dissonance” for this behavior, defining it as explaining away the contradictions rather than seeking to understand them. This is an ever-present hazard when people work together in uncharted territory.

The NSF report suggests how groups indeed can be creative. The key is that interaction with others can stimulate ideas, but then time alone is often required to “allow the socially stimulated ideas to incubate.”

A further observation in the report is that group diversity can be a source of creative ideas. But this is only successful when team members have positive attitudes towards divergent ideas and styles. In an experiment that tested attitudes toward differing ideas within a group, however, Festinger found a problem: Those who believed the others held divergent opinions from their own were less attracted to the group in the first place.

Ethnography

Ethnography is roughly defined as the study of people engaged in group activities. The ethnographer seeks to understand how people’s behavior is a product of how they think about what they are doing.

Everyday Engineering, An Ethnography of Design and Innovation,6 is a collection of studies of engineers at work. The researchers, most of whom had both social science and engineering backgrounds, directly observed the design work on various projects. Their goals were to gain a deeper understanding of the actual creative process. They wanted to use their insights for encouraging the actors to look at their own practices differently.

By observing a number of different projects, the ethnologists found that design doesn't take place in a closed, formally structured location. Perhaps more importantly, it cannot be understood by looking at an organization chart.

Another of their findings was that engineers’ rough sketches, as opposed to formal drawings or models, worked well as a record of the ongoing process. Drawings were used as resources to convince, explain, revise and reach consensus. Observations cited in the NSF workshop support these ethnographic findings. According to the report’s authors, seeking analogies with different projects was often a means for finding a novel solution. This process, they noted, was enhanced when the group used hand-drawn sketches, as opposed to detailed physical prototypes.

Innovation is by definition a break from established practice. It requires solitary thought and introspection as well as interaction with others. It thrives on relationships among different types of innovative minds: idea people, implementation people, management people, and marketing people. They must understand each other and be able to respond to one another beyond the limits of conventional ideas. Engineering innovation can be fertilized by wide reading outside of the field. Management theory, psychology, and ethnology are only some examples. Knowledge of history, politics, sociology, philosophy, pure science and mathematics, and the arts broadens perspectives and opens minds. Innovation requires an understanding of how humans interact and how we interact with technology. Because we are nonlinear, it makes sense that the work of innovation must be multidisciplinary and able to cope with nonlinearity.

References

[1] Collective Genius, by L. A. Hill, G. Brandeau, E. Truelove, and K. Lineback, Harvard Business Review, June 2014

[2] Peter J. Frost and Carolyn P. Egri, in Organizational Reality/Reports from the Firing Line, Fourth Edition, Peter J. Frost, Vance Mitchell, Walter R. Nord, Editors, Harper Collins, 1992, Pp. 449 – 460

[3] Final Report from the NSF Innovation and Discovery Workshop: The Scientific Basis of Individual and Team Innovation and Discovery by C. D, Shun, P. B. Paulus, J. Cagan, and K. Wood http://www.nsf.gov/sbe/scisip/bcs_sosp_wkshp_rpt.pdf.

[4] Groupthink by Jonah Lehrer, New Yorker Magazine, January 30, 2012

[5] Cognitive Disonance: Fifty Years of a Classic Theory, by Joel Cooper, SAGE Publications, Inc. 2007

[6] Everyday Engineering, An Ethnography of Design and Innovation, Edited by Dominique Vinck, MIT Press 2003

Additional readings

A small sample of Festinger’s many published works are:

· A Theory of Social Comparison Practices by Leon Festinger, Human Relations 7, 117 – 140, 1954

· When Prophecy Fails by L. Festinger, H. W. Rieken, and S. Schachter, University of Minnesota Press, 1956

· A Theory of Cognitive Dissonance by Leon Festinger, Stanford University Press, 1957

A paper entitled “Describing the Creative Design Process by the Integration of Engineering Design and Cognitive Psychology Literature,” by T. J. Howard, S. J. Culley, and E. Dekoninck, Design Studies 29, 2, March 2008 includes an extensive reference list.