Requirements vs Architecture

This article will discuss the relationship of architecture to requirements.  From a distance, these two topics appear distinct.  Requirements specify what a system must do.  Architecture describes how a system will be organized and how it will behave in order to fulfill these requirements.  Since requirements describe the problem and architecture describes the high-level solution, then requirements precede architecture, right?  In fact, requirements can be defined without input from architecture, right?  If only life were this simple.

Expert Perspectives:

Let’s start by looking at what some industry experts have to say about requirements and architecture:

Chapter 2 of The IEEE Guide to the Software Engineering Body of Knowledge (http://www.computer.org/portal/web/swebok/htmlformat) says:

“At its most basic, a software requirement is a property which must be exhibited in order to solve some problem in the real world. The Guide refers to requirements on software because it is concerned with problems to be addressed by software. Hence, a software requirement is a property which must be exhibited by software developed or adapted to solve a particular problem. …  By extension, therefore, the requirements on particular software are typically a complex combination of requirements from different people at different levels of an organization and from the environment in which the software will operate.

                    :

                    :

At some point, the architecture of the solution must be derived. Architectural design is the point at which the requirements process overlaps with software or systems design and illustrates how impossible it is to cleanly decouple the two tasks. This topic is closely related to the Software Structure and Architecture subarea in the Software Design knowledge area. In many cases, the software engineer acts as software architect because the process of analyzing and elaborating the requirements demands that the components that will be responsible for satisfying the requirements be identified. This is requirements allocationthe assignment, to components, of the responsibility for satisfying requirements.

Tom Gilb writes, “Requirements give information to the system designers and to a wide range of stakeholders. They state what the stakeholders want the system to achieve.   Requirements can be classified into ‘requirement types’ as follows:

1.     Vision: at the highest level, the future direction for a system.

2.    Function Requirements: what a system has to ‘do’: the essence of a system, its mission and fundamental functionality.

3.    Performance Requirements: the performance levels that the stakeholders want – their objectives. How good?

http://www.amazon.com/Competitive-Engineering-Handbook-Requirements-Planguage/dp/0750665076

Clements, Kazman, and Klein, of the Software Engineering Institute assert that the earliest design decisions have the greatest impact on how well the system will meet its quality attributes.

http://www.amazon.com/Evaluating-Software-Architectures-Methods-Engineering/dp/020170482X

Roy Fielding, co-founder of the Apache Software Foundation and father of RESTful Web Services, states that architecture is a series of approaches. Each approach consists of a set of related decisions that address some concern, and constrain the rest of the system

http://www.ics.uci.edu/~fielding/pubs/dissertation/fielding_dissertation.pdf

Greenfield et al. assert in Software Factories that requirements are not a statement of the problem, but rather statements made by stakeholders about what they would like the system to be.

http://www.amazon.com/Software-Factories-Assembling-Applications-Frameworks/dp/0471202843

What Does This Add Up To?

The first question to address is “What are requirements?”

·       Statements that must be true in order for the system to be suitable

§  Clear            – precise outcome, invariant or specific condition

§  Concise       – each statement addresses one requirement that is independent of the rest

§  Testable      – observable or quantifiable outcome, reproducable

§  Consistent – does not conflict with other requirements

§  Complete   – all requirements needed for suitability are specified

The next question is “Where do requirements come from?”

·       System sponsors  – business, integration, security, schedule, effort/cost

·       System users         – functionality, usability, learnability, performance, administration

·       Authorities            – laws, regulations, contractual obligations, industry standards

·       Mother Nature     – math, physics, chemistry, biology, economics

This leads to the question “What is the relationship between requirements and architecture?”

In Evaluating Software Architectures, Clements et al. quote reknowned aircraft designer Willy Messerschmitt as saying:

“The Air Ministry can have whichever features it wishes, as long as the resulting airplane is not also required to fly.”

In summary, as the following diagram illustrates, requirements and architecture have an interdependent relationship:

 

 

·       Essential outcomes represent the motivation for investing in the development of this system.  Typically, these outcomes exist in the deployment and development environments.

§  In the deployment environment, they take the form of a value proposition – capabilities leading to benefits

§  In the development environment, they take the form of goals or constraints of the development sponsor

·       Discovered challenges consider the essential outcomes and factor in complexities, such as constraints, uncertainties, and interdependencies.  It is the responsibility of the system’s architecture to address these challenges.

§  Often these are forces of nature, sometimes they are externally imposed (laws, etc.).

§  Sometimes, they represent conflicts between essential outcomes

o     Two deployment environments with mutually exclusive requirements/constraints

o     Development environment constraints in conflict with deployment environment value proposition

·       Additional requirements specify detailed acceptance criteria, such as mandatory features or constraints for the delivered system.

As the diagram shows, the higher categories of requirements are binding on the lower categories.  Period.  You cannot have a suitable architecture without understanding why the system is needed, and you cannot understand why a system is needed without grasping how the deployment environment will use it to benefit, and how the development organization will benefit from creating it.  It is important to remember that the external environment always justifies a system.

In summary, every architect is responsible for several things: ·       Understanding the business implications of the top-level outcomes, whether they be: revenue-driven or cost-control, internal or competitive, sales or production. ·       Identifying the complexities and consequences that make up the challenges that will be faced in transitioning from the current state to the desired outcomes ·       Communicating these challenges to stakeholders in a way that reduces conceptual distance, and enables them to understand: o      how their priorities enhance or conflict with others, and o      the critical success factors that have yet to be accounted for ·       Working with domain experts to gain concensus on a plan that Mother Nature won’t veto. ·       Working closely with the product definition and engineering teams to ensure that the detailed requirements and the implementation remain consistent with the architecture.

 

Requirements without Architecture

Without architecture in the picture, high-level external outcomes and constraints often bubble down to use cases, functional specifications, and wireframe UI models.  Now we’re back to me designing the luxury car.  I know exactly how I want the dashboard to be layed out.  I want to be able to change the music without taking my eyes off the road (via speech recognition, of course).  I want feedback on my dashboard to accurately tell me what my current MPG is.   And I want my climate control system to account for temperature and humidity, so my eyes don’t dry out on a long drive.

These are excellent requirements.  They are clear, concise, testable, and most importantly, they are valuable.  What driver wouldn’t want perfect music, optimal fuel economy, and properly moistened eyes?

The trouble is that these requirements assume something.  To paraphrase Willy Messerschmitt, they assume that the car will run.  They assume that it will go from 0-60 in 4.2 seconds and brake from 60-0 in 2.2.  They assume that the vehicle will last for 120,000 miles without anything but routine maintenance.  In short, they assume that these new detailed requirements are consistent with everything that is necessary to offer a luxury car in today’s market.  And they assume that these requirements are complete, that they don’t omit something that is critical to large numbers of potential buyers (is there enough leg room or trunk room, or can the car operate on battery power)?

If you want an emperical test to verify the assertion of what requirements without architectural guidance are like, please conduct the following experiment:

·       attend a requirements review meeting,

·       observe how many reviewer comments address clarity, conciseness, or testability.

·       notice how few reviewercomments address completeness and consistency

Why is this so?  The reason is that it is not difficult for most reviewers to perceive at the level of a single requirements.  However, in order to see inconsistency or incompleteness, it is necessary to visualize the system operating in its intended environment(s).  This is precisely what architecture does.

So, why are people tempted to define the additional requirements before a suitable architecture is defined?  There are a few reasons:

1.     They have a naïve understanding of the uncontrollable forces, and how the affect the desired outcomes.  This has been stated eloquently as, “All things are possible except for those who have to do the actual work.”  

2.    They have had some painful experiences in the past with architects and/or developers, and are reluctant to trust them to do the right thing in this situation.

3.    They have some incentive (compensation, power, prestige, etc.) to achieve a particular outcome or protect a valued position.

4.    They are concerned that formulating the architecture may highlight flaws in the business rationale used to justify the system, whereas completing detailed requirements will make it more difficult to go back.

The first of these is easier to deal with than the others.  Education is possible, as long as people keep an open mind, and you are willing to explain things in terms that they can understand.  If Einstein can explain something as complex as the Theory of Relativity by saying the following:

“Put your hand on a hot stove for a minute, and it seems like an hour. Sit with a pretty girl for an hour, and it seems like a minute”

then I suspect that you can also dig deep and find the ability to express discovered challenges and the approaches needed to address them simply.

The other three reasons are more problematic and depend on specifics of the people involved and the culture of your organization.  However, as an architect, your recourse is clear.  You are the designated representative of Mother Nature, the most powerful stakeholder of all.  Just do the following:

·       get your hands on an early copy of the requirements,

·       differentiate the key outcomes from the details,

·       discover the dominant complexities and their consequences

·       prioritize them and identify the architecture approaches needed to overcome them

·       identify the inconsistencies and incompleteness in the requirements

·       wrangle an invitation to the requirements review meeting, and

·       ask the difficult questions that introduce reality into a naive decision making process

While making the authors uncomfortable is not your goal, it will help to ensure that people who are naive about why architecture matters, will understand this before it is too late to make a difference.

 

Conclusion: Conceptual Distance vs. Systems Thinking

In this article, we have discussed the nature of requirements and architecture, and have concluded that architecture is orthogonal to requirements.  In other words:

·       some requirements – the essential drivers for a system are imposed from outside and drive the architecture

·       other requirements are imposed by architecture in order to overcome the obstacles and risks to satisfy the essential drivers

·       the last set of requirements are the detailed ones, the ones that are necessary to establish that the system meets its acceptance criteria

We also discussed what can happen to requirements without the right architectural influence.  The essential drivers remain the same, but the detailed requirements are often conflicting and incomplete.

In order to have good requirements you need domain experts.  Domain experts are unsurpassed in their ability to identify those conditions and situations where the almost obvious solution fails.  Experts in the right domain can point out for you:

• what road grade that a pickup truck cannot tow a 3,000 pound boat up a hill
• which cities will have late departures, if flights out of New York and Boston are delayed by snow (i.e. given that aircraft and crew will both be delayed)
• which phrases a speech recognition program will have the most trouble detecting
• what areas on the tarmac of an airport will be dead zones for wireless communication

Of course, the trouble is that expertise in one area doesn’t always transfer to another.  The ability to understand an airline’s flight system does not necessarily translate to an accurate understanding of an Internet reservations servers or a database system.  Nor does it even translate into the ability to understand the dialects spoken by server or database experts.

What this does mean is that as complexity grows, the group of domain experts becomes like the United Nations without translators: a collection of special interests, willing to cooperate, but without the ability to understand each other.

This is where systems thinking comes into play.  Architects, by the very nature of their job, are forced to have strong systems thinking skills.  The reason that this is critical is that systems thinking is a learning accellerator.  It is a skill that enables someone to learn new subject matters quickly and well.  This is the same skill that is necessary to translate effectively between domain experts.

It should be clear now what happens during the “Discover Challenges” stage.  The external outcomes and constraints that motivate the system’s construction are on the table.  Now the architects and domain experts prepare to play a game of “Name that Tune”.   Here is are the results we want to achieve and how soon.  How many people do you need for how long to build that system?

The architects and domain experts, burned by over-optimistic responses to these sorts of questions in the past, search for the biggest challenges in the project: deployment environment, technical, and project team.  They prioritize these challenges and debate alternative solutions.  The architects must know which domain experts possess the critical knowledge to answer any question, and they must be able to spot when one approach has potential side effects on other areas (that the domain expert in that area might not recognize themselves).

At the end of this process, the architects and domain experts have collaborated on the solution to a problem than none of them were capable of doing on their own.  They haven’t solved the entire problem.  There is still much work remaining.  But together they have discovered an architecture strategy – the DNA for the eventual solution.  And this DNA came from the best available minds on the various subjects, which means that the approaches are pretty likely to be complete and consistent.