Experiments: Your Net Experiment Score

So far, when it comes to experiments, we’ve talked about what they are, why they’re important, how often we should do them, and how often we should reward for them. Those are all very important pieces of your thinking to refine if innovation is going to become a strand in your organizational DNA.

But let’s take our learning one step further. The culture that I’m a part of loves metrics (perhaps even too much). But what that means is that we try to measure anything and everything that’s missionally meaningful. When I asked my colleague, Dave Smith (who has one of the finest strategic minds I’ve ever known), to help our church think through the right way to track how each of our teams are doing with an ongoing rhythm of experiments so that we could keep it in front of ourselves on our strategic dashboard, he helped us develop what we call our NES—our “Net Experiment Score”.

Since we have the wholehearted conviction that experimentation is the catalytic seed-sowing behavior that sparks innovation, we decided to create and track this new metric. Doing so has injected it into the regular conversation rhythm of our leadership team’s meetings. Rather than just counting the number of experiments, we decided to give different weights to different kinds of experiments using the 4 iZones (see my posts “What is Innovation? Part 2” and “Right-Size Your iZone Investment”).

As you may recall, we agree with others that there are 4 different kinds or zones of innovation. We called these “iZones”—Incremental, Breakthrough, Disruptive, and Game-Changing. These 4 kinds of innovation help us to take into account the varying degrees of progress in both the specific methodology and the market impact. Here is the image to refresh your memory.

Our team gave careful thought to considering the various experiments we’ve conducted over the past few years, categorizing and charting them by the 4 iZones. This helped us to see how well we are distributing our experimental energy across the 4 iZones (NOTE: a 25% allocation to each zone is likely not the ideal. Ask instead what your organization’s season and market conditions currently require). We found that the vast majority of our experimental energy was going into Incremental experiments which would, of course, yield Incremental innovations. That’s neither good nor bad. But it helped us to recognize that we may want to allocate more energy to the other 3 iZones if we want greater impact.

Next, we assigned numeric values to each iZone in order to weight them for their corresponding method progress and market impact. We weren’t scientific in the way we weighted these. The numbers we chose are merely our perspective of the value of each kind of innovation experiment in our unique context (Incremental innovation experiments = 1 point; Breakthrough innovation experiments = 3 points; Disruptive innovation experiments = 5 points; Game-Changing innovation experiments = 10 points). The numeric values you assign may be different; but they should obviously still follow the same progression from Incremental (lowest value) to Game-Changing (highest value).

Out of this exercise, we asked each department team to keep track of the experiments they were doing every quarter—how many, what type, and their weighted value. The idea is that, as a team debriefs an experiment, they would assign it a specific point value based on which iZone it best corresponded to (regardless of whether the experiment was a success or not). In other words, if the experiment was connected to an Incremental innovation, they would earn 1 point; if another experiment was connected to a Game-Changing innovation—and would therefore have exponentially more significant learnings or potential impact—they would earn 10 points).

Every quarter, each team adds up the point values from their various experiments and reports them to the entire staff. This number is that team’s Net Experiment Score (NES). For our context and purposes, we set the NES Value Range at Under-Performing (a NES of less than 6), Healthy (6-7), and Thriving (8 or more). Again, the value range you set for your context may be different.

For our church, in this season, we feel this range allows us to help gauge the experimental effort of a given team. It also allows us to measure the experimental energy of our entire organization. To capture this, you simply add each team’s NES number and divide by the total number of teams. For example, if we have 10 teams in our organization, and their collective scores total 100 points; then our organization’s NES is 10 and, therefore, thriving.

And that’s how the NES or “Net Experiment Score” was born. We’ve found that our NES allows for more nuance, forces us to keep the iZone filters in mind, enables us to quickly track team and organizational experimental energy, and proportionately reward greater risk for impact. This method of tracking experiments may seem complicated. But after you have team leader conversations to set your initial numeric values and NES Value Range, you actually get the hang of it fairly quickly.

One more thought here: we’ve taken our NES and added it to our Innovation Dashboard, which is our highest and most robust way of measuring our innovative energy as an organization (included here as a bonus). This is yet another tool that can be customized to your organization’s innovation strategy.

If experiments truly are the seeds of innovation, and if most teams could benefit from having a more sophisticated way of tracking and measuring experimental energy, then implementing the NES approach in your organization would lead to smarter, more productive, more innovative teams. So what would it take for you to run an experiment on whether the NES would work for you?

QUESTIONS

1. What are other ways your organization has used to try and track experiments? What have you learned about what works or doesn’t work in your unique context?
2. If your team or organization has no method to track experiments or measure their value, do you think the Net Experiment Score (NES) model laid out here would help? Why or why not?
3. Based on the NES Value Range presented above, if you had a team that had a NES of 4, what might that reveal about that team? How would you engage them to increase their experimental effort?
4. Based on the NES Value Range presented above, if you had a team that had a NES of 15, what would that reveal about that team? How would you engage them to reward their effort or ensure that they aren’t over-revving experimentation to the point of exhaustion?
5. How might it help your organization to create your own Innovation Dashboard with your own unique ways to measure and track your overall innovation efforts?