Experiments: Bullets Before Cannonballs

Experiments are the seeds of innovation. And while they are just one of a variety of factors in the formula for cultivating a culture of innovation, they may be the X factor. Here’s why: if innovation can be imagined as a funnel, experiments are the top of the funnel; and the more experiments get poured into the top of the funnel, the more innovative outcomes will emerge from the bottom. So the goal of any team or organization should be to optimize the right number of experiments for their context.

Shifting metaphors with this next idea will sharpen our mental marksmanship. Experiments aren’t just seeds; experiments are also the “bullets” of innovation. You “fire” an experiment to see if you can hit the “target” of innovation. This metaphor for experimentation was made popular by Jim Collins in his outstanding book Great By Choice. Creatively, Collins essentially asks us to put ourselves in the boots of Captain Jack Sparrow during a clash between ships on the high-seas. The wiser captain will first fire one bullet after another in order to dial-in the best trajectory to hit an enemy ship before firing a cannonball which will cost his crew much larger amounts of precious gun powder and lead.

Collins defines bullets as low-cost, low-risk, low-distraction experiments that you fire at your innovation target to dial-in what will help you hit your bullseye—the ideal innovative solution to your core problem. He argues that experiments allow you to “calibrate” your targeting using smaller amounts of resources (time, energy, money, focus). But the goal is, once you have enough evidence that you’re aiming in the right direction, that you then—and only then—fire a “calibrated cannonball” which is a higher cost, higher risk, higher focused—but also higher payoff—effort to blow your target out of the water.

Teams and organizations should never fire uncalibrated cannonballs according to Collins. Doing so is not just terribly wasteful of precious resources. Even worse, it can also result in disaster as you make your company’s ship vulnerable to attack on the high seas of a competitive and hostile market environment. That’s why he contends, “The ability to turn small proven ideas (bullets) into huge hits (cannonballs) counts more than the sheer amount of pure innovation.” In other words, becoming expert experimenters is the most proven path to becoming expert innovators. Fire bullets first. Then, and only then, fire cannonballs.

This idea of “firing bullets, then cannonballs” has been tremendously valuable in shaping our own team’s experiments pathway. We are greatly indebted to Collins. Here are some conceptual nuggets of solid gold from Great By Choice that will add value to your team’s experiments exploration:

• FIRE A BULLET. Then assess: Did your bullet hit anything? Learn everything you can—especially in failure. Bullets give you the opportunity to fail forward. Remember, the point is to fire multiple bullets to progressively zero-in on your target’s bullseye. Then consider: Does your unsuccessful bullet merit re-firing in a different direction or at a different target? Does your successful bullet already merit conversion to a cannonball? Or do you need to fire another bullet to further dial-in? Be sure you understand what the empirical validation is that it worked. Then convert: Once your bullets bring about calibration, don’t wait long to fire the calibrated cannonball. Concentrate your resources and fire. But do NOT fire uncalibrated cannonballs. And terminate bullets that show no evidence of eventual success.

• Conducting experiments (firing bullets) is a form of creative discipline where you build a tangible or conceptual prototype (bullet) to test (fire) and then tweak and repeat as you learn.

• Experiments allow you to scale innovation with consistency since innovation happens more through consistent creative discipline than it does through big-hit, single-step breakthroughs.

• Constantly firing experimental bullets leads you on a search for empirical validation which, when found, protects resources (time, money, energy, focus) and further propels learning. In other words, experiments yield evidence—empirical data—which either validates or invalidates your hypotheses. So keep firing and keep learning. Ask questions like these: “How can we bullet our way to deeper understanding? How can we fire a bullet on this core problem or challenge? What bullets have others fired? What does this bullet teach us? Do we need to fire another bullet? Or do we have enough empirical validation to fire a calibrated cannonball?”

• Teams that build their reserves and then resolve to experiment, even into and through failure, are teams that fail forward. “Greatness requires the Churchillian resolve to never give in, but it also requires having the reserves to endure staggering defeats, bad luck, calamity, chaos, and disruption.” Get it? Experiments are the stepping stones to greatness.

• The breakthroughs that come from the empirical validation that come from the creative discipline of experimentation show that innovation is NOT “predictive genius” (being able to predict the future and then profitably position yourself for what’s coming).

Innovation demands developing experimental expertise. That’s what firing bullets first is all about.

What’s the point of this cluster of posts on experiments? Innovation requires the skills of a farmer, a gamer, a pirate, and a sharpshooter. And so whether you look at experimenting as planting seeds or playing games or firing bullets, the point is… experiment.