November 5, 2018

Using Entropy to Measure Software Maturity

  —A look at using information theory to measure software.

In Using Entropy to Measure Software Maturity, Kamiel Wanrooij explores a metric based upon changed files as a measure of software maturity. Kamiel calls this entropy and uses Shannon’s information theoretic definition. I applaud Kamiel’s desire to create a metric to measure software change.

Kamiel’s thesis is that software files can be viewed as providing information regarding the ability of the software to support change. That is, a project comprising \(n\) files contains \(\log_{2}{n}\) bits of information. This model ignores the changes within a file. It relies on a measure of the number of changed files.

The utility of this measure is limited. In C and C++, the file is the unit of compilation. I’ve worked in environments where we’ve adhered to a function per file rule and where all related elements get added to the same file. And environments with an annual update of the copyright, which affects every file in the project. Your mileage will vary considerably depending upon how you use files.

The value of this measure lies in the charts wherein the magnitude of the files changed is presented. You can monitor this and investigate the peaks and valleys. (Peaks for the reasons discussed in the article; valleys because someone might add a lot to a few files.) The use of a logarithmic scale needs to be weighed against a count–perhaps you want the peaks and valleys amplified.

Files changed is a response to the change being made to the project. In statistics, a response depends upon one or more explanatory variables. The article points out that coupling is an explanatory variable.

The implication that high coupling causes a lot of files to change. It might. It might not. It depends upon how files are used in the project.

The best rationalization for using Kamiel’s proposal as a software metric is that it works best in an environment where the following are true.

1. Files and entities of interest positively correlate. For example, you enforce a rule of one function/method/class per file.
2. There is a tendency to increase the number of files as the structure of the project unfolds. For example, you enforce a rule where refactoring adheres to the principles mentioned in Kamiel’s article.
3. There are mitigations in the process to prevent adding a lot of changes to a single file. Such changes would represent a valley in the charts and the logarithmic scale would diminish this.

In all, I think the metric proposed by Kamiel has value but needs to be acknowledged as a response. Understanding, nay controlling, the explanatory variables driving the response is paramount to getting high value from this measure. Even then, directly measuring the explanatory variables is better.

The Reddit discussion on this article. I agree with the notion that averaging over time is a poorer choice than using a histogram or heatmap to measure change. Using a heatmap has the advantage of highlighting areas where change is consistently high.