When I started applying the lessons from my “Scrum for Knowledge Work” in product development teams with deadlines, it became clear right away that some things were working better than others. The structure made it possible to implement Dr. Ward’s vision for “learning before commitment.” But teams were overwhelmed with the number of questions they had to answer before they could commit.

They had little in the way of knowledge that they could reuse, since most of it was locked inside the brains of the more experienced engineers or in inaccessible systems. Even if they did intend to reuse a technical solution, there wasn’t much available to explain how it worked or why the original team had made their design decisions. So, it seemed like they needed to learn everything.

The first boards had too many questions in the backlog, and then once they made it from the backlog into a Sprint, the questions took a long time to close. This led to a “snowplow” effect where the numbers of unclosed questions grew and grew until the system collapsed. I could facilitate a team through this only by helping them ruthlessly prioritize, but how did they know which questions were the most important ones to answer?

The Right Questions to Answer

As we experimented with different ways to filter the questions down, I saw that the potential impact of a question on a program’s direction was an obvious selection criterion.  But how could we know whether or not a given question had high impact or low impact, especially if it wasn’t obvious? After looking at one complex program in particular that had nearly snowplowed its way past a major milestone, I noticed that the questions that made the most difference were usually paired with a decision that followed.

I had started calling these questions Knowledge Gaps by then: a gap in the team’s knowledge that they needed to close before they could commit. For that program, I reverse-engineered the decisions that would be made after a Knowledge Gap closed, and then asked the team what would have happened in a typical development program? That was the second breakthrough.

The Second Breakthrough:  Better Decisions to Eliminate Risk

On a typical program, these decisions would be made, usually a lot earlier, without the knowledge the team was trying to build. But it would be a decision on shaky ground, one that a good program manager would treat as a risky decision. If it was a major decision, it would have to be revisited if the risk of a wrong decision materialized. And if that decision had too many Knowledge Gaps — leaving the original decision on very shaky ground — it probably would be revisited later, when more information came in.

Yet it was treated all along like a final decision, so every revisited decision was treated like an exception to the normal process, which created a lot of disruption. Good program managers added buffers and margins to deal with the inevitable problems, but all too often, the buffers and margins would get eliminated in a misguided attempt to go faster, or consumed by the first few decisions that needed to be changed.

The Impact of Revisiting a Final Decision

When a major decision needs to be changed late in development, that change comes with a lot of cost. The decision has been embedded into the product, with other decisions made that depend upon it. The fixes to the problem are usually not quick and simple. They may require doing something that significantly delays the program, or increases the per-unit cost. The team may have to cut a desirable feature, or compromise on the user experience in order to get the product out at all — much less on time.

I’ve come to think of decisions like that as long, slow learning cycles. We make decisions early in development that don’t get validated until much later in development. When we find out that we have to revisit the decision, we sometimes have to loop back to much earlier stages of development to fix the problem: adjust the requirements, specs, or product design.

Teams sometimes try to fix these “execution challenges” by changing the execution process in some way, or even worse by locking down requirements to avoid late design changes. But these fixes don’t work because the problem is not rooted in the execution process, and normally it’s not an option to just ignore the needed change — at least if the team wants to release a viable product.

The only way to eliminate this problem is to eliminate these long, slow loopbacks at their source: decisions made without the knowledge needed to make the decisions with confidence.  When teams can do that, they run into many fewer issues during the execution phases that slow them down.

They don’t even have to get it perfect — they just have to eliminate enough of these late-found defects to avoid overwhelming the team’s capacity to make changes. One of these is not usually enough to drive a product development program over the edge, but a handful, especially if they are all interconnected, can trigger enough design instability to tip the product over into an endless series of “build-test-fix” cycles with no end in sight.

The way to eliminate these long, slow loopbacks is to make these major decisions at the right time, with the right people and the best available knowledge.

Focused Knowledge Gaps Build Knowledge for Better Decisions

I started using the term “Key Decision” to refer to the High Impact / High Unknown decisions that tended to trigger long, slow loopbacks, and then we began to find Knowledge Gaps by first identifying these Key Decisions. That immediately drove the team to work only on the Knowledge Gaps that would make the most difference for the program as a whole: the ones that would lead to better Key Decisions.

The other decisions had much lower risk, either because they had lower impact, or they were better understood. Those decisions could be made more casually without increasing risk. In fact, it wasn’t worth the team’s time to improve the quality of these decisions because they had no material effect on the program as a whole.

That breakthrough went a long way towards eliminating the “snowplow” effect of too many Knowledge Gaps that made too little difference to the program. It was a lot more clear what Knowledge Gaps needed to be closed in order to make better decisions, and ultimately a better product.  The team was not overwhelmed with too many unanswered, yet unimportant questions to close.

Decisions At The Right Time With The Right People

When we coupled that with a directive to make these decisions at the right time, we found we could delay many of them, and that they were even less likely to be revisited if they were made as late as possible. The extra time meant that Knowledge Gaps could be closed more completely and that dynamic situations had more time to stabilize. In the meantime, everyone knew that the decision wasn’t final, and made other decisions accordingly, preserving flexibility to accommodate the right decision.

Finally, we began asking teams to be more conscious about who needed to make the decision, and especially which stakeholders needed to be engaged for a decision to be a good one.

When the first teams began making their Key Decisions at the right time, with the right people and the best available knowledge, we started seeing performance improve: teams hit their milestones on time, and ultimately released the product faster. They spent less time churning in late development on revisited decisions. They were able to make better decisions about cost, features and performance, which meant the products did better in the market. And they were able to do all of this without any additional resources — the development budgets and teams were the same.

Much Better — and Much More Potential

The teams were happy because they were getting better results, but I could see that the process still wasn’t working as well as it could work. Teams were still spending too long in the learning phases and arriving at Key Decision points without the knowledge they needed. They could spend too much time figuring out what they needed to learn in any given Learning Cycle.

The third breakthrough gave the teams the visibility they needed into the flow of knowledge so that they could maximize the value of the time they had to learn.