I’ve written about McCall’s Software Quality Model and applying it to software quality assurance to create measures throughout the lifecycle. The team and I are trying to employ this model and have some challenges.

• Should there be a “time-to-release” quality objective? Let’s define time-to-release as a function of cost and time that is minimized. For example, \(people \times hours \times (dollars / hour)\).
• Are there quality factors that can (should) come after “time-to-release”? Let’s say there are quality factors that we want to monitor but not optimize.

Should time-to-release (e.g., something measured using time and cost) be a quality factor?

A quality factor is (FACTORS IN SOFTWARE QUALITY):

A condition or characteristic which actively contributes to the quality of the software.

Quality factors affect cost, whether measured in terms of time, dollars or people.

… all factors are related to a normalized cost to either perform the activity characterized by the factor or to operate with that degree of quality. For example, maintainability is the effort required to locate and fix an error in an operational program. This effort required may be expressed in units such as time, dollars, or manpower.

So every factor has a cost and the units of cost need to be normalized so they are easy to compare.

The rules for quality factors.

The following rules were used to determine the prime set of quality factors:

• a condition or characteristic which contributes to software quality,
• a user-related characteristic,
• related to cost either to perform the activity characterized by the function or to operate with that degree of quality,
• relative characteristic between software products.

Bullet four implies that cost and time are quality factors. It connects managing quality to managing time or cost. Table 2.4-1 in FACTORS IN SOFTWARE QUALITY, titled “Grouping of Software Quality Factors to Achieve Unambiguous Set” includes cost and time.

Elsewhere, FACTORS IN SOFTWARE QUALITY explains that cost and time are not quality factors. They form the basis for correlating metrics with various levels of quality.

The quality triangle is another way to view this relationship. Since the quality model focuses on managing software quality. You can choose only one of cost or time as the second focus of management.

McCall’s model ignores quality factors based upon cost and time because it includes provision for normalizing all quality factors using time or cost. You can’t use both. Including time-to-release as a quality factor isn’t supported by the model.

In all, strictly applying McCall’s quality model means that cost or time is normalized across all quality factors. Not doing so means your model is less coherent (quality factors are not comparable). If your quality factors are not comparable you will have more difficulty determining how to allocate resources.

As to the second question, “Should time-to-release include notions of quality factors that come before and after it?” the answer is no. If you insist, take a look at Time-to-Release – the missing System Quality Attribute.

You can view this as a simplification of McCall’s model: spend more resources on those attributes above time-to-release and less resources on those below. (Note this article points out that related ISO standards ignore time-to-release, so what’s proposed in this article isn’t recommended.)

The lesson here is that time-to-release is a useful approach for managing McCall’s quality factors. It’s not best practice and that means it introduces other challenges. The main disadvantage introduced by time-to-release is the inability to normalize the costs of quality factors. That’s a bad thing if you define quality as something that controls cost. (You should.)

McCall’s quality model was published in 1977 (see A Framework for the Measurement of Software Quality). It’s significant because its one of the first models to capture user experience and developer perspective.

Another document FACTORS IN SOFTWARE QUALITY complements what’s in the framework with more detail.

McCall’s Software Quality Model provides a nice high-level introduction into the relationships between model components. It doesn’t touch on the quantitative aspects of the model. The model provides a framework for measuring software quality that affect user and development facing properties of the software.

The Model

The model is built upon three product pillars:

• Transition captures how well the product can be adapted to new environments.
• Operation captures how well the product fulfills its specification.
• Revision captures how well the product can be revised or changed.

Each pillar ties represents an activity occuring on a software product throughout its lifetime. These pillers are use to create a collection of quality factors aligned those activities. A quality factor captures behavioral and nonbehavioral requirements.

Product Operation

Operation provides a behavioural representation of the product. Behavior includes quality factors like correctness and efficiency.

Product Transition and Revision

Transition and revision provide a non-behavioral representation of the product. Transition includes quality factors like maintainability and flexibility. Revision includes quality factors like portability.

Quality Factors and Criteria

Quality factors are positively influenced by one or more quality criteria.

A quality criteria is software-directed metric (actual, quantifiable measurements) that relate to the factors.

For example, a quality factor for simplicity impacts reliability. Reliability is measured in terms of the extent a program can be expected to perform its intended function with required precision.

The same criteria can impact a number of factors. For example, portable code may degrade efficiency.

Some factors positively or negatively impact other factors. For example, correctness positively affects reliability but generality might negatively impact efficency.

Employing the Model

A key point in McCall’s paper is that the metrics for quality criteria are measured throughout the development of the product’s software. This is in contrast to testing which only identifies the frequency of error.

Furthermore, the quality measure process must be applied to the requirements and design phases of the product. Quality metrics are predictive in nature and oriented toward the development phases rather than toward the finished system. They are a tool employed to reduce the cost of correction before the final product is delivered.

Measurements

Keys to measurement success:

• attributes are predetermined and a consistent method of mesurement must be employed
• a relationship must be developed between the product and its application
• such measurements are not absolute

Measures are then a rating for a quality factor. This rating must include a function mapping each quality criteria into the quality factor.

Sarah Mei has a tweet:

We think awful code is written by awful devs. But in reality its written by reasonable devs in awful circumstances.

A lot of the comments focus on developer ability. My favorite response is the truth table–surely done in jest.

Developer ability is a narrow view of the challenge here. People make mistakes, exercise poor judgement and create less than ideal outcomes. This happens under good circumstances as well.

I think Sarah’s point is that environment and constraints affect code quality. I asked her:

Am I correct that your notion of awful circumstances includes organizational (management, process, etc.) down through to the personal relationships between team members?

She responded:

Yeah. There is a very narrow slice of it that any of us has control over.

A narrow slice of control. I’m hopeful.

It’s important to recognize the socio-political environment and psychological elements.

• Management support.
• Team power structure.
• Personal relationships.
• Individual stress, burnout, frustration, capability and ability to contribute.

It starts with people. The narrow slice of control revolves around how people treat each other and the choices we make to respond to our challenges. The trick is to maximize the number of good choices we make.

I have hope that awful circumstances can be improved. I believe empowered people can make decisions that improve outcomes. The trick is to listen, act and repeat.

A narrow slice of control might be all you need.

I’m facing a challenge wherein I need to improve my team’s focus on architecture and design. My perspective on this is that their approach is ad-hoc. Team members complain that decisions are made without discussion. The results are suspect and limiting, or least don’t fit my definition of a successful outcome.

My team uses Scrum and struggles with design. My first experience with Scrum left me with one unanswered question: “Where is design occurring?”. I think the short answer to that question is that it wasn’t occurring in a way that promoted a shared understanding of the goals.

In taking on architecture and design I’ve had to rethink my position on how to approach it. Most of my thinking is around identifying what to focus on and how to communicate it so the team gets a shared understanding of the objective.

The first thing I did was look at the product requirements. There was an implicit notion of an architectural quality objective in many requirements. For example, people would talk about performance or reliability but often just use the word itself as if it’s presence conveyed something important. What was missing was an exploration of what performance or reliability really meant. What was missing was tangible activity around these objectives both in the form of requirements and in design.

I started presenting quality objectives as the “goodness” we wanted from the architecture. It was a cheap sales pitch. No one will sign up for badness. It was a good idea in that it created a focal point for discussion. I could ask questions like “Are we getting the goodness we need?” and even “How can we enhance the goodnes we need?”.

The funny thing is that people responded positively to “goodness” over “quality objectives”. People became engaged on finding and identifying “goodness”.

The most interesting thing about goodness is that it turned out to be a good communication tool for stakeholders. Those people outside of the Scrum and Development teams.

Using “goodness” provided a way forward and to begin a conversation about what it meant to have a reliable or performant product. It provided a way to change the perspective on our requirements elicitation that put architectural quality objectives in the same class as functional requirements.

I’ve discussed this essay in a couple of places:

• The Mythical Man-Month (Worth Reading Again) renegotiate your schedule, not your deliverable. An optimal schedule uses as many people as there are independent subtasks (and no more).
• Hatching a Catastrophe discusses how a lack of hustle leads to a lack of urgency around small schedule slippages. The challenge is to create a sense of hustle and urgency in a team when these occur.
• The Mythical Man-Month (Revisited): discusses how people use this essay to deflect from good work breakdowns. The challenge then is to educate people on the need for deeper analysis.

My reading of these essays is that they provide useful and valuable insights even for an Agile team. And yet, the team and I are challenged by these things.

A team member is interested in discussing these essays. Bravo, I say! I’m hopeful this person will provide insight that the team embraces.

An ideal outcome for a discussion on these essays should include:

• Recognition that the question of whether you can add more people to a task is reasonable until such time you’ve identified all the substasks. Shutting down discussion on this question prematurely is one way to misuse these essays.
• Recognition that a slip of a day is a signal and not something to be afraid of. Delivery oscillates–sometimes you pull ahead of the projection; sometimes you lag.

An Agile team should be comfortable using swarming to address the issue of optimal subtasks. A Scrum team is ideally 4-7 people. This number of people doesn’t introduce a communication barrier, since you are all in stand up and it completes in 15 minutes.

An Agile team should be willing to help each other so that day-by-day adjustments can be made to adjust for progress changes. The stand up is an ideal place to discuss whether you are able to help or need help on a task.