Product teams face a challenge when justifying investments in a product opportunity, despite ambiguity and uncertainty. Making hard choices, such as this, is a core part of building great products. Before a go/no-go decision is made on the investment, the product team is typically asked for their confidence level in winning with the opportunity. 

While confidence level is often expressed as a percentage, context on the percentage is useful in communicating with stakeholders. A rich shared vocabulary is useful for this. The US legal system offers a vocabulary with its “burden of proof”concept. The product team has the burden of proof, and the go/no-go decision is like a guilty/not-guilty decision. Three of the many standards of proof in the US legal system are “beyond reasonable doubt” (to prove that a defendant committed a crime), “clear and convincing evidence” (for fraud cases), and “preponderance of evidence” (for civil lawsuits such as contract/copyright violations). 

Examples of using burden-of-proof vocabulary

• “There is a preponderance of evidence that opportunity X is worth pursuing. An experiment with select customers will make this clear and convincing. And a plan to resolve issues uncovered in the experiment will make this opportunity real and beyond reasonable doubt.”
• “We plan to collect evidence beyond reasonable doubt for opportunity X. But for opportunity Y, we plan to collect only clear and convincing evidence because we believe it has lower market and feasibility risks.“
• Similar to above, for hypothesis testing: “We plan to use a 95% confidence level for experiment X to get evidence beyond reasonable doubt. But for experiment Y, we plan to use a lower 80% confidence level to get clear and convincing evidence.“
• “We plan to interview a representative set of customers, do a competitive analysis, and test two concepts to get a preponderance of evidence to decide whether or not to allocate a team to explore a potential opportunity.”

Translating to confidence level percentages

A study by Duke Law School in 2016 translates these standards to probabilities. 124 judges were asked questions of the form, “What level of certainty must you have to find a criminal defendant guilty beyond a reasonable doubt?” This resulted in a 90.1% level of certainty for “beyond a reasonable doubt” , 73.4% for “clear and convincing evidence”, and 54.4% for “preponderance of evidence”.

The burden-of-proof vocabulary allows for a nuanced understanding of the confidence level beyond simple percentages. Percentages are the vocabulary, for example, in pharma which is regulated by the FDA.  

 

A rich shared vocabulary can help product teams build conviction – pun intended – towards a go/no-go product opportunity. 

PS: Check out more articles on building products. I write to pay it forward and to clarify my thinking.