Quality Attributes, Contexts and Challenges

Introduction

Over the past several months, I have observed several product managers, architects and engineers misapply quality attributes, quality attribute scenarios, contexts and challenges.  While these concepts are closely related (even to the point of some overlap), the nuances driving their distinctions are important.  In this post, I will try to clarify these distinctions and how they affect some common applications.

Quality Attributes and Quality Attribute Scenarios

Quality Attributes and Quality Attribute Scenarios are concepts that were popularized by the Software Engineering Institute in the early 2000’s.  In their book, Evaluating Software Architectures, Rick Kazman, Paul Clements, and Mark Klein observe:

1. Software products (and products, more generally) can be evaluated according to two primary criteria, based on the utility they provide:

           Functional Capabilities:   What can the product do?

          Quality Attributes:          How well does the product do them?

For example, automobiles have engines, steering and brakes (capabilities).  They also have fuel economy, turning radius and deceleration times (quality attributes).

2. Architecture represents the set of earliest design decisions which combine to have the greatest impact on a product’s quality attributes.

These early design decisions can also affect or preclude functional capabilities.  For example, the microphone in a cell phone might enable phone calls, but it might not provide a clear enough signal to enable speech recognition.

3. However, a quality attribute is not precise enough to evaluate architecture.  Several additional factors are required to evaluate an architecture decision:

           Stimulus:                         What event triggers the assessment and how can it be specified and/or quantified?

           Environment:                   What conditions is the system operating under?

           Response:                        What is the system’s expected outcome and how will it be quantified?

Is 25 MPG good fuel economy for an automobile?  It depends.  Is it highway or stop-and-go city driving.  Is the automobile carrying a driver, or is it pulling a boat and trailer and carrying 4 adult passengers?

These extra factors turn a quality attribute into a scenario, and enable a more precise assessment of that quality attribute.

The Quality Tree

Kazman, Clements, and Klein express the important relationship between quality, quality attributes, and quality attribute scenarios as a tree.  Figure 1 below shows a partial example – the ellipses show areas likely to be expanded in a full tree.

Figure 1:  Partial Automotive Quality Attribute Tree

The three levels of this quality attribute tree are:

       Root                          Identifies the system of interest: in this case, the product type (automobile) for which quality will be modeled.

      Quality Attribute   Represents a general characteristic that has value for a system stakeholder (fuel economy, deceleration, etc.)

      Scenario                         Stimulus, response and environmental characteristics used to assess a named “test case” for a quality attribute.

Quality Attribute Scenarios are the “atomic element” of architecture evaluation, and Quality Trees are a perfectly good way to organize them.  However, it is important to realize that the Quality Tree is just a view, and it is only one of several useful views. 

The motivation for this assertion can be found in the following statements:

1. Figure 1 contains several repeating characteristics, including:
• Traffic conditions (city/highway),
• Pavement type,
• Terrain slope,
• Vehicle speed (or change),
• Vehicle direction,
• Passenger count, and
• Payload carried by vehicle

            Why are these characteristics interesting?

2. These variables are reused in different quality attributes

           Why are these characteristics reused?

3. The values of these variables follow certain patterns

           What is the significance of the patterns in variable values?

Challenges, Value and Quality Attribute Scenarios

Certain environmental characteristics drive the degree of difficulty in achieving a quality attribute.  Others barely matter.  For instance, fuel economy is very sensitive to city versus highway conditions.  Relatively constant 55-65 MPH speeds enables engines to perform at their best.  Frequent stop and go doesn’t.  On the other hand, fuel economy is not significantly affected by the current weather.  Fuel economy varies with gross vehicle weight (GVW); however, 3 additional adult passengers and 1,000 pounds of cargo in the trunk probably might increase a car’s GVW by 30-50%.

A challenge is a constraint or uncertainty that has a significant impact on the ability of a design decision to realize one or more quality attributes.  The most obvious kinds of challenges are found in environmental characteristics, such as highway vs. city conditions, weather, or terrain slope.  However, some challenges are introduced by other design decisions.  Curb weight (the weight of an empty vehicle) represents at least 60% of gross vehicle weight and is entirely driven by automotive design.

Value is a subjective perception of stakeholder (or many like-minded stakeholders) of the utility of a product or service.  The use of the phrase “subjective perception” here is both significant and intentional.  Benefits are objective.  Value is perceived.  MPG is a benefit.  Some people could care less if their vehicle gets 10 MPG.  Other people are appalled that anyone could have such disrespect for the natural resources of the planet they inhabit that they would even consider driving a vehicle that gets less than 30 MPG.  Subjective perception?  You betcha!

Now, let’s tie this discussion of Quality Attributes, Value, and Challenges together into a tidy package.  Figure 2 below illustrates the key relationships:

      Drivers                   are the variables which dictate the value of a system.  They may be referred to as goals, objectives or requirements.

      Challenges             are constraints and uncertainties which make it difficult to satisfy a system’s drivers.  They may be referred to as barriers, obstacles, risks, or critical success factors.

      Decisions               are choices made about a system intended to satisfy its drivers.  They include financial, marketing, architectural, design, resource allocation or project decisions.

      Test                       are physical or conceptual experiments that are run in order to validate how well a system of decisions overcomes a set of challenges in order to satisfy a set of drivers.

Figure 2:  Drivers, Challenges, Tests and Decisions

The pattern in Figure 2 should be obvious to anyone who has been involved in product development:

• Requirements are the drivers.
• Decisions are what make up the software implementation.
• Tests are the set of cases in the QA test suite, and
• Challenges are the obstacles, risks, and safety hazards to be verified.

Figure 2 is not limited to executable systems.  The architecture evaluation process described by Kazman, Clements and Klein applies before system implementation:

• Marketing and business goals are the drivers.
• Decisions are the early decisions and approaches in the architecture.
• Tests are the set of quality attribute scenarios to evaluate, and
• Challenges are the obstacles and risks caused by external conditions and internally-imposed policies and constraints.

Why Context is Critical?

In practice, it turns out that challenge and value are highly-correlated concepts.  The external temperature has little impact on fuel economy, but sub-zero temperatures pose a significant challenge for starting a car.  They thicken motor oil, limit evaporation of gasoline, and weaken a car’s battery.  This example underscores the fact that challenges and value are not isolated entities, but rather occur in related clusters.  Context is the term we use to describe a collection of stakeholders who share a similar set of perceptions, priorities, and desired outcomes and are subjected to a similar set of conditions and forces.¹

Context is a very important unifying concept.  Why?  Because it shifts the focus from the product or system to its intended use (and the environment of intended use).  Theodore Levitt stated this quite clearly when he observed:

“People don’t want to buy a quarter-inch drill, they want quarter-inch holes.”

Products and services are perceived to have value to the extent that they solve problems!  Beauty is in the eye of the beholder.  This is why there are tens of millions of people who share by their Apple iPad or Samsung tablet, while a similar volume of people can’t imagine what all the fuss is about.  The same products have very different perceptions of value.

This is why context is critical.

Context links challenges with perceptions of value.  A group of stakeholders face similar conditions and seek similar outcomes, and have similar priorities.  It should come as little surprise that these individuals (or organizations) are very likely to value potential solutions in a similar way.

There is a reason that suburban families with young children value SUV’s, minivans, and crossover vehicles.  How else can you transport kids and their sports equipment to their soccer, football or ice hockey games?  There is also a reason that people who own boats or campers tend to buy pickup trucks.

Context and Quality Attribute Scenarios

Contexts are important for three reasons:

1. Value, as perceived by a stakeholder, is rarely tied to a single quality attribute.  Few car buyers are addicted to reliability or fuel economy.
2. Even if a stakeholder’s value model is dominated by one quality attribute (such as standing-start acceleration, top-end speed, or engine volume), it is rarely dominated by a single set of conditions (i.e. scenario).
3. Perceived value changes over time.  Examples include significant changes in income or family status.

In other words, a context can be viewed as a container for a set of related:

• Like-Minded Stakeholders
• Value Models (i.e. what the stakeholders desire and their priorities)
• Situations (conditions with respect to a point in time and space)
• Quality Attribute Scenarios (links between situations and value models)

This abstraction is powerful because it provides a means to organize scenarios and compare assessments across:

      Value Models    Stakeholders in similar situations who want different things.

      Time                 The same group of stakeholders at different points in time.

      Assumptions     Different assumptions about the same group of stakeholders in the same future time period.

Conclusions

Quality Attribute Scenarios are an atomic unit of architecture formulation and evaluation.  For a given aspect of quality, they capture stimulus, environmental conditions, and expected outcomes.  A complex system might contain thousands of quality attribute scenarios.

Good design of quality attribute scenarios for a system is a complex problem, as complex as designing a complete test suite for the same system.  While architectures are higher level than implementations, architectures have many more unresolved issues (resulting from many more degrees of freedom).

The selection of outcomes to quantify and exogenous variables to control represents a difficult choice.  In the automotive example above, why focus on traffic conditions, pavement conditions, terrain, cargo payload or other variables?  Why not focus on temperature or air quality?

Contexts and challenges are excellent tools for framing and organizing quality attribute scenarios.  Contexts organize like-minded stakeholders, their situations, and the challenges they face.  Challenges identify how conditions and constraints create obstacles to desired outcomes.  Just as contexts contain quality attribute scenarios, they also contain challenges.

As a result, contexts are the keel of a sailboat.  They are a fundamental tool for ensuring consistency between:

• Stakeholders
• Value models,
• Challenges, and
• Quality Attribute Scenarios.