Product Talk

Make better product decisions.

Brainstorming: More Questions Than Answers

Originally published: by | Last updated:

Continuous Discovery Habits book coverHave you heard? My new book Continuous Discovery Habits is now available. Get the product trio's guide to a structured and sustainable approach to continuous discovery.

There’s been a lot of confusion around brainstorming these days. First, there was the New Yorker article by Jonah Lehrer claiming that brainstorming doesn’t work. Scott Berkun wrote a rebuttal arguing Lehrer’s logic was flawed. Twitter exploded with comments. I’m going to try to make sense of it all.

What is brainstorming?

The term was introduced by Alex Faickney Osborn in his books, Your Creative Power and Applied Imagination, from the late 1940s / early 1950s. He argued that brainstorming, a group process, generated more ideas of higher quality than thinking of ideas individually. Osborn, an advertising executive from a top firm, was writing from experience. He was not an academic researcher nor did he have data to support this claim. But because of the reputation of his firm, his books gained in popularity and brainstorming spread throughout the business world. To be clear, I’m not criticizing Osborn, but I make this distinction because I’ll be tackling some academic research below.

Osborn argued in order for brainstorming to work, people had to overcome social inhibitions and focus on generating a large number of ideas. He outlined the following rules of brainstorming to support this idea:

* Focus on quantity.
* Withhold criticism.
* Welcome unusual ideas.
* Combine and improve ideas.

Some of these might sound familiar, the first two in particular, as these are what we typically associate with brainstorming.

Why brainstorming might not work?

Jonah Lehrer authored an article, Groupthink: The Myth of Brainstorming” in the January 30, 2012 issue of The New Yorker, where he claimed, “there is a problem with brainstorming. It doesn’t work.” He then went on to describe a study conducted in 1958 that compares the creative output of brainstorming groups vs. individuals. The article doesn’t provide a citation, but fortunately, Imagine: How Creativity Works, Leherer’s book, does. The study he refers to is by Taylor, Berry, and Block and is described in “Does Group Participation When Using Brainstorming Facilitate or Inhibit Creative Thinking?”, published in the Administrative Science Quarterly.

To be clear, this is not the same study that Berkun criticizes in his rebuttal. I’ll tackle that one next.

The Taylor, Berry, and Block study compares the creative output of groups of 4 people with those of individuals. It also compares the “real” groups’ output with “nominal” groups’ output. The “nominal” groups were constructed randomly from the individuals’ outputs. As to be expected, the real groups outperformed individuals by themselves. However, the nominal groups significantly outperformed the real groups. Basically, combining the efforts of four individuals working alone significantly outperformed the output of four individuals working as a group.

This study is well designed. It used three different problems – each seemed reasonable to tackle with brainstorming. It evaluated creative output on quantity, originality, and quality. The quality criteria seemed applicable to each of the problems in the study. The groups were familiar with each other and had experience working together. The experimenter acted as facilitator when criticism arose. The participants were experienced with the brainstorming process. The methods seem sound. I really wanted to find a problem with this study. But I couldn’t. Also, the statistical difference wasn’t slight. It was large. It does seem to support Lehrer’s claim that brainstorming doesn’t work.

The authors of the study suggest two reasons for why this might be. First, they argue that even with deferring judgement, social inhibitions exist. People are less likely to suggest odd ideas in front of a group rather than alone, speculating that group participation in and of itself is inhibiting. They also speculate that group participation may impede the variety of ideas. The participants might be subject to group think and be susceptible to following the same line of thought.

Both of these ideas resonate with me. As an introvert, I much prefer to think through something on my own. I have a whole slew of “crazy” ideas in my notebook that would never see the light of day if other people were involved. I’ve definitely been steered in a particularly direction based on someone else’s suggestion before I’ve had time to think through all the options.

So what’s with this other study? The one Berkun rips apart?

Lehrer describes a second study, the work of Nemeth, et al. in his New Yorker article. Berkun rips it apart. Like Berkun, it’s not clear to me that this study is even asking the right question. It compares the creative output of three groups: one is given no instruction, the second is given Osborn’s brainstorming rules, and the third is given Osborn’s rules but told it’s okay to criticize each other’s ideas. In short, the second group outperforms the first and the third group outperforms the second. The authors conclude that criticism and debate outperform brainstorming.

I don’t like this study. The groups didn’t know each other ahead of time. They weren’t familiar with brainstorming techniques ahead of time. The difference between the second and third groups seems minimal. Who knows what the first group actually did in practice? It just doesn’t seem like a good study. Since the 1958 study was so strong, I’m focusing my discussion on that and excluding this one from the conversation. This doesn’t seem to cause any harm as it doesn’t really introduce anything new to the conversation.

So let’s assume that brainstorming in the Osborn sense doesn’t work. Why do so many creative experts still swear by it?

Tina Seelig includes it inGenius: A Crash Course on Creativity. Tim Brown and Tom Kelley, both of IDEO, write about it respectively in their books, Change by Design and The Art of Innovation. Daniel Pink writes about it in A Whole New Mind. What’s going on?

It’s possible brainstorming does work.

There are a lot of reasons why even the Taylor, Berry, and Block study might be wrong. Tina Seelig argues brainstorming is hard. It takes practice. The IDEO folks sing a similar tune. It’s hard to reserve judgement when someone raises a silly idea. It’s even harder to generate more ideas when you think you’ve already come up with a viable solution. Maybe it takes practice for a brainstorming group to be effective.

Or maybe it takes facilitation. The experimenter did point out when people criticized each other’s ideas. But for those of us who have facilitated brainstorming sessions, we know there’s a lot more to it. To keep a brainstorming session running smoothly, you need creative prompts. You need to encourage the group to explore and build on each idea. You need to reframe the problem when the group gets stuck. This is a skill that likely wasn’t present in the study groups.

Or it might take the right people. Tom Kelley, in The Art of Innovation, talks a lot about getting the right people in the room. You need diversity of expertise. You need broad representation. You need different styles of thinkers. Some research suggests that the ideal group size is 5-8. The study only had groups of 4. Could this be part of the problem?

Without doing a more exhaustive survey of the research and / or conducting more research, I don’t have the answers to these questions.

Or we might just be confused.

We might just want brainstorming to work. On the surface, it makes sense. You and I will probably approach the same problem differently. If we combine our efforts, we’ll get more divergent thinking. This is true. But we might not get there together. It is possible that our perception of the group experience is interfering with the objective results. How often do we actually compare the results of a brainstorming session with the collective individual results of a similarly-sized group?

Or it’s possible that we are confusing the value of brainstorming with the value of creative association. Getting a room full of people to free-associate around a problem seems like the same thing as what we see innovators do all the time when they combine two unlikely concepts to form an innovative solution. But these aren’t the same thing. An engineer and an animator serendipitously having a conversation in the hallway at Pixar is not the same as putting the two in a room and asking them to brainstorm.

Taking in a variety of diverse inputs and making creative connections between them is a very different activity than a forced brainstorming session. The former takes advantage of the subconscious mind the latter is a cognitive process. Is it possible that we are merely confusing these processes?

So what’s next?

When I first started writing this post, I was hoping to read a couple of studies and form an opinion. I thought this would turn into a “how to run an effective brainstorming session” post. But far from it, this journey has raised far more questions than provided answers. I suspect this will not be my last post on this topic. I’d love to hear you thoughts.

Do you value IDEO’s experience and creative output over the academic research? Have you seen other studies that help answer some of these questions? Do you plan to keep running brainstorming sessions? Why or why not?

References

A lot of different pieces influenced this blog post. Note: All links to books, both here in the references and above in the post, are Amazon Affiliate links.

Get the latest from Product Talk right in your inbox.

Join 41,000+ product people. Never miss an article.

    Comments

    1. Avatar for Susan Barrett KellySusan Barrett Kelly says

      Thoughtful post. So many of us use tools like brainstorming without much thought to their design. Sent an article via Twitter from McKinsey Quarterly from last spring that also featured steps to better brainstorming.

    Popular Resources

    Recent Posts