Does Agile Scrum Kill Innovation? (Iterative Problem Solving)

Scrum is designed to solve interrelated problems common – but not unique – to software development projects: project scope and estimation.  

Do these problems also exist in innovation projects? Will the Scrum solutions work for innovation projects as well? Short answers: Yes and Yes. 

Applying Agile Scrum methodology to the process of Innovation allows the product team to define the scope of the project more clearly, identify emergent work more easily, estimate the work to be done more accurately, and demonstrate value to customers and stakeholders more frequently.

Let’s walk through what Scrum looks like within traditional software development and then pair it with common problems faced by innovative teams.

How Scrum uses emergent prioritized work to solve problems

Scrum developed ways to address the problem of project scope (the total amount of work needed before the project can be considered done). Since software can do so many things, it is hard to limit our needs. Therefore, project scope in software development has always been a huge problem.  

In software development projects, project scope usually starts out too big and only increases as the project progresses. Without understanding the project scope, it is hard to know when a software development project is ‘done.’ Therefore, projects often end when they run out of money or time, rather than being declared ‘done.’

Scrum addresses the project scope problem through ’emergent prioritized work,’ collectively referred to as the backlog. The backlog starts with currently known work and collects emergent work as the effort proceeds.

In Scrum, one person (the product owner) prioritizes the backlog; this makes the backlog manageable. In each iteration, the team faces a limited set of high-priority work and understands what it means to declare that set of work ‘done.’

When an iteration is completed, the project must always be functional so the users can touch and feel it for flaws or errors. Each time, the product owner shows the results to the users and asks their opinion on business value created or development errors (often some version of ‘what do you hate most’). In this way, the biggest possible mistake or waste of time is limited to a single iteration. If the project is off the rails, it can be reoriented before too much damage is caused. Thus, the process creates ideas for additional, emergent work which can be added to the backlog. 

The product owner prioritizes the backlog items according to business value. In this way, the product owner can compare the business value created by an iteration to the cost of that iteration. If there is no combination of work items with a total business value larger than the cost of an iteration, then the project is finished. 

Innovation projects have similar problems.

How do we decide what the most pressing problem is?

How can we know when we’re done (or even when we will be done)? 

Why Agile Scrum is best for innovation 

We build the result a little bit at a time, ensuring the work we’re doing fits within the whole idea of done and to make sure we’re also collecting any emergent work. This incremental approach helps solve scope problems in innovation projects, just like it does in software development projects.  

Another problem iteration can help solve is estimation. Everyone wants to know how much a solution will cost and/or when it will be done.  

Any time we discuss something in the future, we engage in estimation. Given what we understand in the present, we make predictions on how the future will play out. Unfortunately, the only truth we can predict is that humans are terrible predictors of the future (flying cars, anyone?).

In fact, we are so terrible at prediction that our only defense is to think small. Scrum helps solve this by using an iteration small enough that it allows:

  1. Product backlog items to be estimated reasonably well
  2. The risk of an estimation error to stay at a manageable size. 

Over a handful of iterations, the team’s productivity per iteration (velocity) becomes apparent. The product owner can then estimate the completion of the known backlog with a reasonable level of accuracy. If this estimate is over budget or schedule, the backlog can be pruned, and/or the organization can (carefully!) work to increase the team’s velocity. 

When you’re involved in innovation efforts, consider a lightweight Scrum process involving:

  1. A backlog of needs
  2. Iterations of work focused on a few specific needs from the backlog
  3. Regularly showing how your solution actually works (even if it works poorly).

This way, you can focus on the most valuable work while effectively collecting emergent work and managing costs and schedules.