May 22, 2015

Why Most Unit Testing is Waste (A Look at Test-Driven Development)

  —A look at Test-Driven Development in James O. Coplien's article on waste in unit testing.

I reviewed “Jim Coplien and Bob Martin Debate TDD”, to better understand Coplien’s position on Test-Driven Development (TDD) in “Why Most Unit Testing is Waste”.

In defining TDD, Martin states that it is infeasible for a software developer to call themselves professional if they do not practice TDD. He goes on to cite the laws of TDD:

1. Don’t write a line of production code unit you have written a failing unit test.
2. Do not write more of a unit test than is sufficient to fail. (Not compiling is a failure.)
3. Do not write more production code than is sufficient to pass the currently failing unit test.

In order to properly understand Martin, it is important to understand his use of “professional”. A professional is an expert. [1] Later, Martin says that he thinks it is irresponsible for a developer to ship a line of code that has not been executed with a unit test.

Essentially, expert software developers test their work using unit tests.

Coplien’s problems with TDD are two-fold.

1. The use of TDD without an architecture or framework.
2. The use of TDD without an architecture or framework destroys the GUI.

Both problems have the same root cause: a poor architecture. Coplien provides examples where a lack of domain knowledge contributes to these problems. A lack of domain knowledge has no bearing on the usefulness of TDD.

TDD can be used to drive architecture. Driving architecture is one thing. Bootstrapping it is another matter entirely.

Coplien and Martin agree: do some up-front architecture, but don’t knock yourself out. Let executing code inform future decisions.

How much up-front architecture is required?

Coplien says that a 2 million line program should have constructors and destructors in place and enforce important relationships between objects and that these relationships be supported by tests. And you should have executable code for this implementation within 30 minutes.

Coplien’s tests are not unit tests. Coplien defines a unit test as an API test. It tests a subset of the state space of the API arguments. It’s a heuristic. He suggests “Design by Contract” is a better choice.

Design by Contract (DBC) ties an implementation to business requirements. TDD obfuscates this because the emphasis on unit tests can make it difficult to connect functionality to business requirements. It's not clear that the use of TDD implies causality.

Martin’s position on DBC is that he prefers unit tests tied to production code instead of contracts embedded within the production code.

Martin implies that something being used (TDD) is better than something that is not being used (DBC). I agree. Lack of use implies low utility but so does misuse. Low utility does not mean that the ideas in DBC are invalid. Unfortunately, the DBC verse TDD discussion doesn't go very far before the session ends.

This debate provides clarity on bootstrapping an architecture. Bootstrapping is hard and the transition from exploring the problem space and defining an architecture to developing executing code involves many tradeoffs. A balance is required. Perhaps poor judgement creates an imbalance that results in the use of TDD too early.

I like the notion that assertions provide a nice coupling between the semantics of the interface and the code itself. This provides a clear advantage over a separate unit test to enforce these semantics. Employing assertions for pre-,  post-condition or invariants is a clear win. However, using assertions does not eliminate the need for unit tests. Both should be used to create advantage.

[1] See Professionalism and TDD (Reprise). TDD currently plays a significant role in professional behaviour. Experts exhibit professional behaviour.