June 3, 2021
Pareto Charts and Retrospective
—A look at qunatifying your retrospective data.
I’ve been engaged in retrospective for multiple teams and have always run them with the same principles:
- create an environment where people are free to discuss the good and poor outcomes.
- review our current best practice and revise if we can identify mitigations to prevent poor outcomes.
- drive towards an action or decision on each topic discussed.
The most recent set of changes to our retrospective are discussed in Changing Defect Management to include Root Cause. Recently, I was told that I’m doing retrospective wrong.
The argument was that we shouldn’t be fixing anything without a Pareto chart of the issues. I discuss Pareto Charts in several articles:
- Data Measurement and Analysis for Software Engineers which identifies a Pareto chart as a good tool for analysis.
- Revising the Pareto Chart which reviews the statistics behind the Pareto chart. It includes a Monte Carlo simulation that shows the value of acceptance levels on each Pareto category.
- Another Look at Revising the Pareto Chart which includes the source code for the simulation in the previous article.
This article explores the relationship between the Pareto chart and a retrospective. It also considers the value of small wins and frankly, makes an argument that the current structure of the retrospective is better.
The chief characteristics of retrospective is that the team reflects on their performance over the last sprint and seeks to identify things they should
- start,
- stop and
- continue doing.
A retrospective includes consideration of
- processes,
- people,
- relationships and
- tools.
A retrospective actions items immediately so corrections can begin during the next sprint.
A Pareto chart is a tool for statistical process control among a large set of quality factors. I’ll define large as statistically significant.
The debate I entered into was whether the immediate correction or a Pareto chart was the correct way to go to improve quality. The argument I made was that because of the way the Pareto is structured, all I have to do is monitor the issues that come in through retrospective and track trends. (I was monitoring issues and had the data to prove it.) There is no difference here, except that I don’t wait for a period of time to act on the information.
Put simply, if the same issues keep coming up in retrospective, then they will get actioned. The more frequently they come up, the more frequently they get actioned. All the Pareto will show is that we actioned the correct issues.
So I ask you, gentle reader, what’s the difference?
Well, there is one difference. The timing of events could skew the frequency of events so that in theory I could act on issues that are less important because they occur less frequently. The key issue here is the time frame for data collection. And guess what? The Pareto will show the same thing.
Well, there is a second difference. You can argue that you need a statistically significant number of samples in order to identify the correct action to focus on. I can buy that, but then isn’t the argument that something is better than nothing while you build up this sample? That’s an argument for small wins.
So I’m back to where I started. There is nothing wrong with the recommended actions in the Scrum Guide to action retrospective items. Tracking the data as it comes in through retrospective is a great idea. Not being able to generate a statistically significant data set isn’t a problem if you do it intelligently. Take advantage of small wins.
