Invisible Requirements

by Charlie Alfred and Garrett Thurston

Introduction

The story has been told too many times to count. A firm spends in excess of a million dollars and 18 months to develop an innovative new product, but after months of development, a requirements problem cripples the effort.

Sometimes, it’s a case where the requirements are there, but the product developer and the development sponsor interpret them differently. Other times, requirements that were written down imply other requirements which weren’t. In other cases, the requirements are new, triggered by external events or revised assumptions about the future.

Once manifested, invisible requirements tear deeply into the work that has already been done, rendering core assumptions invalid. In the best case, significant redesign and rework must be done. In the worst case, it’s less expensive to start over than to try to salvage what has been created. In either case, somebody is held accountable for the mess.

Requirements Role in Product Definition

Any successful product development effort fundamentally depends on the following:

• Where and Why – The product needs a strong fit with its intended market(s) and the environments where the product will be used.
• What – The product must have a clear set of requirements, which provide a complete set of acceptance criteria.
• How – The product must have a clearly defined reference model that provides conceptual integrity by explaining how the product is put together, how it works, what its limits are, and how it can be adapted for different situations.
• Who, When, and What If – Finally, the product must have a well-defined plan for what resources are needed, how their efforts will be coordinated, and what steps should be taken if unexpected problems surface.

Each of these characteristics of successful product development efforts is important. Together they form the backbone that holds everything together. An ever increasing focus on the quality of requirements, particularly in the Aerospace and Defense industry, is providing significant gains in productivity and lower costs. To drive these effective requirements, industry has found that the input to the requirements process, the high-level architecture, is critical in formulating requirements that specify clearly all necessary functionality, while constraining the implementation as little as possible. Indeed, architectural consideration—such as deployment contexts, constraints, and desired though- not-required behavior—are invaluable in the early stages of systems and system-of-systems development.

Therefore, while an increased focus on requirements is necessary for successful product development, it is not sufficient. Industry has found that a thorough treatment of the higher-level architecture provides the requirements effort the analysis and data necessary to effectively drive the rest of the development life cycle.

The Problem with Solution Orientation

Coming to solutions (requirements) too quickly often times overlooks potentially more beneficial solutions. To illustrate this, consider the Jefferson Memorial.

Several years ago, excessive erosion of the Jefferson Memorial was noticed. A brief investigation identified excessive cleaning as the cause. Since the memorial must be kept clean, more investigation was necessary. Bird droppings were identified as the culprit, so actions were taken to have fewer birds around.Eventually, however, someone asked why the birds were such a problem with the Jefferson Memorial and not the others. Another study determined that the birds frequented the memorial, not for love of Jefferson, but for love of the many tasty spiders that made their home there. Probing further, the spiders were thriving because the insect population was proliferating.Finally, understanding that the insects were attracted by the memorial lights at dusk and dawn identified the ultimate solution. Turn off the lights. Initial solutions driven by quick decisions by memorial managers (i.e. powerful stakeholders) provided expensive ill-suited solutions for an ill-understood problem. The root cause and final solution requirement were well hidden, only brought to light by extensive and time consuming trial and error solutions. Each required solution inappropriately framed the problem, missing the associated hidden causes and final necessary requirement.

Requirements and Architecture

Most product development processes today are organized into multiple phases (for example, conceptualization, feasibility, planning, development, etc.) with formal gates[i] connecting adjacent phases. Often organized in a top-down structure, later phases tend to elaborate on and enrich the decisions of earlier phases. Increasingly, life cycles, while maintaining the gated structure, are formulated as iterative, spiral, or Agile processes— rather than a simple waterfall. Traditionally, the life cycles are initiated with the development of product requirements.

The classic riddle “Which came first, the chicken or the egg?” has a product development version: “Which comes first, requirements or architecture?” Traditionally, requirements preceded architecture. After all, you need to know what before you can decide how. As products grow more complex, with increasing numbers of interconnected, interoperable systems and subsystem; and an ever-present drive to support reuse and product lines, the requirements equation becomes more complex. In first examining architecture—exploring product drivers, possible solutions, and architectural constraints and opportunities—organizations are able to establish a solid foundation on which requirements can be effectively derived. Hence, for most products, the market need and initial stakeholder requirements must be analyzed before the product concept and associated requirements takes shape.

Since a well-defined architecture provides a solid framework on which requirements can be established, the resulting requirements tend to be complete and consistent. In contrast, when requirements are specified as input to architecture, this foundation is shaky or non-existent, and results in an incomplete, overly constrained set of product requirements.

Indeed, even when a product is to be derived from a similar product (an earlier generation from the same company or a similar product from a different company), the requirements are still vulnerable, without first examining the architectural implications of both the similar and target products. For instance, assume that the environmental forces and system capabilities of an electro-mechanical control system are similar from one generation to the next. In this case, the number and type of I/O points, as well as the way-signal validation, failure logging, and reversion occur are also likely to be similar— but does this tell the entire story?

The risk for using existing reference models is that they may not match the current problem—or there may be more to the current product’s story. In the previous example, changes in system capabilities, performance, or deployment contexts may invalidate the reference model. In contrast, the need to implement a product-line architecture might drive additional capability and extended or alternative ranges. The first obstacle is discussed later on in this paper in the section on “Inconsistent Requirements.” The second issue is a prime example of an invisible requirement.

What Are Invisible Requirements?

An invisible requirement is one that is captured incorrectly, expressed vaguely or left unstated (i.e., assumed) before development begins on the part of the system that depends on the requirement.

Therefore, in the example in the previous section, the business need to lower life-cycle costs in subsequent product iterations drives a product-line architecture approach. If this need is not clearly understood before product requirements are initiated, this invisible requirement will wreak havoc downstream, as extensive rework is required to establish the product line—or, worse still, a product is produced that doesn’t support product lines and in turn the business need to lower-life cycle costs over the medium and long term has also been traded away.

Many invisible requirements are seemingly innocuous. For example, suppose a system raises an alarm whenever a network connection is dropped. Later, when the hardware and software are integrated, the requirement must change to become three attempts to reestablish the connection before raising the alarm. The coding change is minor, with certifiable software development, the impacts on the requirements, design, and verification (reviews and tests) can be significant, and can impact key delivery milestones.

Hence, invisible requirements are often as innocuous as a carbon monoxide leak. Colorless and odorless, they strike with deadly force. They hit the hardest when their impact has a wide-ranging effect throughout the entire system.

Obscured Tradeoffs

Requirements provide both guidance for development decisions, as well as the criteria to verify the suitability of the end product. Ironically, this dual responsibility turns out to be one of the sources of invisible requirements.

Verification criteria must be precise. The system either passes the test or it fails. The same is not entirely true for development guidance. Charles Kepner and Benjamin Tregoe published a book called The New Rational Manager[ii]  in 1981. In this book, the authors discuss a method called Decision Analysis: an objective, rational process for making complex decisions among many alternatives.

Kepner and Tregoe’s method is straightforward: evaluate each decision alternative against a set of objectives. These objectives fall into two categories: must and want. Must objectives are binary criteria which answer the “whether or not” question. If any must objective is not satisfied, the alternative is rejected. This sounds a lot like the verification process. If the system passes all of the tests, it is considered suitable.

Kepner and Tregoe proceed to make an interesting point. They say that must objectives only filter out the unsuitable alternatives and narrow the field down to the real contenders. Want objectives address the “how well” question and collectively identify the best alternative. Want objectives define acceptable ranges for criteria and a way to map this range to a measure of utility.

For example, a must objective might say that response time must be 200 milliseconds or less for the current planned hardware configuration. A want objective to support future hardware configurations might necessitate a tighter response time, say 100 milliseconds. Therefore, though not necessary for the current effort, if possible within the current projects cost and schedule constraints, a tighter response time is preferred for future instantiations.

When each want objective is prioritized relative to the rest, adequate information exists to make informed tradeoff decisions.

Most requirements specifications that we have encountered take the form of a lengthy list of must objectives. While this approach is necessary for evaluation, when it comes to guiding development, it is a car without a steering wheel. The information garnered in defining a robust system architecture provides that steering wheel, allowing good tradeoff decisions and mitigating risks.

Want and must objectives relate to the degrees of freedom for the downstream processes. A specification with all must objectives doesn’t have any leeway, i.e., no degrees of freedom. In contrast, allocating some objectives as wants provides context and flexibility. In other words, want objectives add degrees of freedom, must objectives take them away.

Therefore, when system verification is performed, an unambiguous set of must requirements are essential. When the solution approach is formulated, however, a much different approach is needed, since the set of requirements which support architecture and design, should take a different form than those which support verification. All requirements, however, must be consistent.

Inconsistent Requirements

Consistency is an essential property for requirements, and cannot be evaluated for any requirement in isolation. Consistency requires two or more items and a context that defines rules for their coexistence.

One simple form of inconsistent requirements is two requirements that contradict each other. Another is when two requirements cannot coexist given external constraints, such as the laws of physics. For example, a plane shape and wing design may not have the aerodynamics to remain airborne at some minimum air speed.

Requirements reviewers need a proven reference model to assess consistency. This reference model is composed of scientific laws, engineering discoveries, and past experience, especially with next-generation products. However, past experience with prior-generation products has blind spots, since next-generation products often introduce major new technological capabilities, which can have a significant impact on the architecture and change the rules used to evaluate consistency.

Cognitive Biases

Cognitive biases are mental errors caused by heuristics that can compromise accuracy for speed and ease of judgment.[iii] Cognitive biases are very different from cultural bias, organizational bias, or bias resulting from one’s own self-interest, since they do not result from any emotional or intellectual predisposition, but rather from subconscious mental procedures for processing information.[iv]

One area that illustrates how product development can be affected by cognitive bias is the development of manned versus unmanned systems and their associated reference models. The engineering challenges for manned and unmanned aircraft certainly have a great deal in common; for example, both must take off, fly around, and land. They are, however, fundamentally different in that the pilot in unmanned aircraft is either automated or remote. As a result, many of the rule-of-thumb considerations for aircraft development, from development considerations to certification regulations, must be carefully reexamined to examine the innumerable cognitive biases established over the past 100 years of flight experience.

This illustrates why it is so easy for cognitive bias to creep into people’s decision processes. Those who are intimately familiar with the issues and challenges of one domain unconsciously carry a schema of associations, assumptions, and conclusions into the other domain. Often, these schemas frame[v] their thought processes and judgments so strongly that representatives from the two groups with disparate cognitive biases often have difficulty understanding each other.

For example, several years after a project award, at the tail end of the certifiable development cycle, an Avionics control program manager innocently shared with the system integrator—their customer’s customer—that they, of course, would receive well regulated power from the aircraft power bus. The project manager was confident that the Avionics control would perform against its documented requirements, as long as his long-standing assumptions on power quality were met.

You can imagine the surprise and chagrin as it came to light, during the systems testing phase, that power levels quality should be expected to vary significantly, and that it was the Avionics control system’s responsibility to detect and adapt to this variability.

This invisible requirement resulted from different, unspoken assumptions between prime and sub-contractors about the essential nature of the operating environment. Eventually, the realization of the invisible requirement forced a complete redesign of the system (including hardware, software, and supervisory functions), leading to millions of dollars of unanticipated costs and highly visible schedule delays.

Fortunately, one need apply suitable means to address cognitive biases. This is most effectively done through disciplined use of analytical tools (several of which are introduced briefly towards the end of this paper) that help break down the fast-and-frugal heuristics in favor of a rigorous analysis process.

As mentioned above, cognitive biases can form roadblocks between groups with disparate cognitive biases. These challenges often occur on a broad scale when participants from business, marketing, sales, engineering (Systems, Hardware, Software), and maintenance roles introduce their own subtle cognitive framing biases when establishing product requirements.

Incomplete Requirements

Complete requirements are necessary and sufficient to solve the problem they were intended to solve. As a consequence, any requirements which are missing, implied, or unstated are invisible, by definition.

How can you be sure that a set of requirements are complete? Gathering all of the requirements you have specified is not enough. Completeness cannot be evaluated by looking inside the system. To evaluate completeness, it is necessary to look outside—to the containing system.

To evaluate completeness of requirements, we must understand the product’s stakeholders, its external systems, and the governing laws of math and science. We need a reference model (or robust system architecture) that describes how these entities will drive, influence, or limit the product. Some of these factors, like gravity and electricity, are predictable and measurable. Other factors inject uncertainty and provoke mayhem, such as unclear market demands and volatile functional expectations. Still other factors have voluntary opinions and perceptions, ranging from predictable to surprising.

Driven by uncertainty and the likelihood that things will change over time, the goal of complete requirements is elusive. However, an acceptable outcome is achievable, but only if the analysis of completeness shifts from a focus on the product to a focus on the context in which the product is used. This shift in focus enables the product creator to “walk in the shoes” of the buyer and end user, ask more meaningful questions, and imagine what additional sources of value a solution could provide.

The challenge of invisible requirements increases as the size and complexity of the system increases, sometimes further complicated by additional supply-chain elements.

For example, consider a plumber coming onto a construction site and taking care of his part of the effort, but while so doing damaging or invalidating what the carpenter or electrician had already done.

In complex aircraft systems with distributed supply-chain elements, such occurrences frequently occur. Furthermore, such systems frequently involve competitive bidding, which, while driving the price down, can often lead to various suppliers not asking key questions for fear of providing development insight to their competitors. The result: not all requirements and constraints are identified as a solution is formulated.

Context Dependency

Many products have several contexts of use. A context of use (context for short) occurs whenever one or more of the following aspects, outside the product, can vary in a way that creates serious stress for the product’s design.

Conditions                          Constraints or uncertainties which differ from one context to another. Conditions can be physical, such as physical operating environment, including temperature, humidity, and electromagnetic interference. Conditions can be functional, for example, radio or cellular network access. Conditions can also relate to variances in product regulations, for example, export control (EAR/ITAR) and airworthiness (FAA/EASA or military).

Situations                           These are forces that have a strong time component. For example, climate variables in most locations vary by season, with inevitable weather driven challenges in a flight-reservation system. Furthermore, these can be difficult to predict, in the case of weather, or almost entirely predictable, in the case of peak holiday air-traffic.

Perception                         Whenever stakeholders form a group (such as users), perceptions tend to differ within subgroups. Often these perceptions will vary with conditions and situations. For example, airport flight operation managers in cities with cold winters are likely to place a higher value on ice-level detectors more than those in cities with warm winters.

Requirements issues related to context dependency can have devastating consequences. By analogy, if implicit or missing requirements are a virus, then multiple context requirements are the H1-N1 strain, as the needs of an entire (critical) stakeholder group may be ill served or ignored.

Unfortunately, context-related issues are common. One of the biggest reasons is that product development companies tend to fastidiously focus on the near-term market opportunities, and treat other contexts as distractions to be dealt with later.

Multiple context systems[vi] often occur in the product-line arena, in which many related products (or models) intentionally share common assets to increase ROI by systematically containing development costs. Note, however, that single products often have multiple contexts, too. A telltale sign of a multi-context single product is a large number of configurable options. Consider the following well-known example of an invisible requirement resulting from multiple contexts.

The original Jeep was a versatile, light-weight four-wheel drive ground transportation vehicle used during World War II by the U.S. and Allied Armies. After the war, the Jeep was redesigned and commercialized for civilian use. The primary context of use was “off road enthusiasts”, i.e, drivers who enjoyed driving on rugged, uneven terrain.Ground clearance, for users in this context, was a critical product quality. It’s not hard to understand why tearing off a gas tank or exhaust system on a tree stump or rock outcropping 50 miles from nowhere was considered to be a bad idea.Over time, the SUV increased in popularity, spawning many new makes and models. The primary context of use shifted from off-road enthusiasts to urban and suburban drivers. However, the suspension and ground clearance did not change.One of the unfortunate side effects was a significant increase in the number of rollover accidents involving SUVs. As the context of use changed with the paved surface and higher speeds, the higher-ground clearance, with its associated higher vehicle center of gravity didn’t. Off-road drivers typically drive only 35-40 MPH, on flat terrain with few obstructions. City and suburban drivers often drive 65 MPH, sometimes on wet or icy highways. Therefore, when an SUV driver has to brake sharply and swerve to avoid a vehicle stopped in traffic, the high center of gravity becomes the driver’s enemy.

The usual motivation for multi-context systems is to increase ROI across many products by leveraging commonality. However, experience with product-line architectures has shown that properly managing the differences between contexts matters most. The key implication of this is that if you have, or even think you may have, a multi-context system that you want to leverage with a single architectural strategy, it is imperative that you investigate the contextual differences early, before the architecture hardens. Skipping this activity is virtually certain to create invisible requirements that may not reveal themselves for years—and when they do, like the SUV, they will reveal themselves at the least opportune time and in the least opportune way.

Agile Methods

Agile Methods, such as SCRUM and XP have emerged during the past 10 years as a way to cope with invisible requirements. These methods shun detailed system architectures and requirements specifications (known as NBDUF, for “no big design up front”). They maintain that too much is unknown and too much will change. In other words, stakeholders don’t really know what they need until they gain experience with the product. Early designs are bound to be filled with incorrect and unnecessary features, so effective products emerge only after series of successive iterations often involving multiple course corrections, each of which help to evolve the product in the right direction.

Clearly, with certifiable system development and the high-costs associated with rework, one of the biggest threats stem from “what you don’t know that you don’t know.” Traditional waterfall processes often try to deal with an uncertain future using insurance, in the form of speculative requirements and designs. Agile methods follow the YAGNI (“you ain’t gonna need it”) mantra and avoid purchasing this insurance until it is proven to be necessary.

However, neither approach has a crystal ball. Both are forced to handle surprises. Agile approaches rely on refactoring to restructure implementations that have been ambushed by invisible requirements. Waterfall methods do this also. There are two critical factors in the comparison.

1. When an adjustment is required, how much existing development do you need to rearrange? Early in the development life cycle, Agile methods, with their focus on staying lean, may have an advantage. There are fewer components and their interdependencies are less constraining. Here, the cost of refactoring is more manageable. Over time, as the size and complexity of the product increases, the structure hardens, and changes become more risky and expensive.
2. Of the opportunities for up-front decisions, how many will be correct versus how many will be misguided? If you are going to be wrong, which side do you err on?

Agile proponents maintain that when you make the decision up front and find out it is inappropriate, you must lug it around like the proverbial albatross. Waterfall proponents argue that if you postpone the decision, refactoring later on is much more costly and risky than getting it right the first time.

This argument is virtually impossible to win in the absolute. For safety-critical systems, major refactoring late in the process can introduce unacceptable risks, as well as large cost impact, due to changes which ripple through the entire process. For information systems that support human activities and decision making, the pendulum swings the other way.

Synthesis as a Stabilizing Force

The question is not whether early or late decisions are superior, or how establishing firm requirements up front lines up against with the ability to adapt quickly. Product developers want to make good decisions both early and late. They want to anticipate what is likely to occur and react quickly to the unforeseen. In this volatile, complex, uncertain world, only the prescient can do this reliably.[vii] How can the rest of us do this better?

Russell Ackoff, former Professor Emeritus at the University of Pennsylvania discussed two forms of thinking, analysis and synthesis, which were known to the ancient Greek

philosophers.[viii] Analysis, the act of decomposing bigger things into smaller pieces to understand how they fit together, has been the dominant mode of thinking since the Industrial Revolution. Synthesis shifts attention to the containing system to grasp why things are the way they are, and what forces and cycles hold them in place.

In the sections above, we have discussed several forms of invisible requirements:

1. Those which define tradeoffs.
2. Those which make other requirements inconsistent.
3. Those which are missing or unstated.
4. Those which are correct in one context, but not in another.
5. Those which emerge unexpectedly due to unforeseeable events

In reality, the first four elements in this list (and some of the fifth) are not invisible at all. They are merely obscured, hidden in plain sight. To identify and understand them, it is necessary to shift our way of observing systematically—to put aside our cognitive biases, employ the right analytical tools, and ask the right questions early.

We rely on analysis to comprehend what buyers and end users say they want, to heed the requirements that regulators impose, and to demonstrate how the product will meet the revenue, margin, and ROI hurdles. However, these things fail to give us enough insight into “why” they want what they want, or why these regulations exist, or why the business needs to meet these particular financial objectives—and when we lack understanding of why, that is precisely when invisible requirements emerge.

Analytical Tools and Methods

There are any number of analytical tools that can be combined to help mitigate the issues associated with invisible requirements, including: Product Strategy Framework, Multiple-Context Analysis, and Road mapping.

Product strategy framework[ix] is a method that rationalizes the core product vision with that of the strategic vision. The formation of these visions is more proactive than the SWOT[1] analysis, popularized by Michael Porter[x] in the 1970s and ‘80s. With a product strategy framework, the strategic and product vision are aligned by considering the organizational core competencies (value chain), financial plan (economic model), technology trends/strategy, product strategy, and market trends/competitive strategy.  This methodology allows you to exerting more control trying to get where you are headed as quickly as possible.

Multi-context analysis, discussed above, is a formalized examination of the variability associated with a product’s (or product line’s) context. This analysis first identifies the various deployment

contexts, taking care to serve not only the present needs, but also downstream considerations. The context needs and constraints are documented and prioritized, such that the critical contexts are addressed by the always limited time and cost considerations. Variability analysis, examining the ranges of the product requirements necessary to serve all targeted contexts comes next, driving firm verifiable requirements into the development process.

Road mapping[xi] looks at the long-term plan as both a mechanism to create and maintain alignment among business, marketing, and technical stakeholders and decision makers.  Roadmaps provide different views into efficiently bringing the right product to market, from concept, through product development, to market introduction/final-delivery. Not so surprisingly, these views mirror many of those examined in the product strategy framework. Roadmaps ensure that all stakeholders are served by maintaining an ever-evolving roadmap, supporting informed tradeoffs up and down the organizational chain. Furthermore, developing a consistent set of roadmaps aligned with those of customers, ensure both short-term success and long-term utility across the entire customer-base.

Conclusion

Invisible requirements aren’t really invisible. They are just hidden from view, because of the way that we look at them. Here are some basic tenets that should guide your thinking if you want to maximize the number of invisible requirements you notice:

1. Customers don’t buy from you because you offer a high-quality well-engineered product. They buy because it solves a problem for them or provides value in another way.
2. Mother Nature is one of your biggest stakeholders. Your product can only do what the laws of math, science, and economics permit it to do. The demands of your customers, your executives, and shareholders must be reconciled with environmental forces.
3. Contexts are the natural combination of the first two items. It is essential to approach product development in a context-sensitive way. Earlier in this paper, we observed that contexts occur whenever conditions, situations, or perceptions change in a way that fundamentally changes the problem. It should have been evident to SUV manufacturers that off-road enthusiasts and yuppies had very different conditions and perceptions, and, as a result, there was a good chance that the same product might not be appropriate for both contexts.
4. Challenges are an important tool for connecting combinations of customer situations. Challenges are risks and obstacles that make it difficult to realize value. Challenges occur within our product and drive our architecture decisions, but they occur inside of our customer’s contexts, too. The ability to identify and solve major challenges faced by a customer context is the foundation of a successful context.
5. A best practice in architecture strategy is to focus architecture decisions on the highest priority challenges. The highest priority challenges are the ones that hold the key to unlocking the most value, and are the most difficult to solve. This approach is critical to making the best possible tradeoff decisions for a product. A clear understanding of “why” clarifies the reasons that certain features and capabilities are more valuable than others, and helps to explain what kinds of changes in conditions or situations will change how value is perceived. Likewise, a clear understanding of contexts can help to illuminate opportunities for how technologies can be applied to overcome challenges, which historically have blocked the realization of value.

An architecture strategy based on overcoming well-prioritized challenges acts as a sailboat keel to stabilize development. This type of architecture, available for a relatively small investment of time and money, provides a solid foundation, for both waterfall and Agile development methods.

---

[1]Strengths, Weaknesses, Opportunities, and Threats

---

[i]Cooper, Robert, G., Edgett, Dr. Scott J., and Kleinschmidt , Dr. Elko J., Optimizing the Stage-Gate® Process: What Best Practice Companies are Doing – Part One Reference Paper #14

[ii]Kepner, Charles H., Tregoe, Benjamin, B. The New Rational Manager, Princeton Research Press, 1981, pp. 77-100. The original version, The Rational Manager was published in 1965.

[iii]Bostrom, Nick, November 18, 2007, http://www.overcomingbias.com/2007/11/towards-a-typol.html and http://en.wikipedia.org/wiki/Cognitive_bias

[iv]Richards J. Heuer, Jr. “Psychology of Intelligence Analysis”, CENTER for the STUDY of INTELLIGENCE Central Intelligence Agency, 1999

[v]Tversky, Amos, and Kahneman, Daniel, 1981. “The Framing of Decisions and the Psychology of Choice.”

Science 211: 453-458.

[vi]Alfred, Charlie, “Multiple-Context Systems: A new Frontier in Architecture”, The Microsoft Architecture

Journal, Volume 23, http://msdn.microsoft.com/en-us/ff476942.aspx

[vii]Singer, P.W. Wired for War, Penguin Press, 2009. Addresses what Kurzweil terms “The Law of Accelerating Returns.” (see http://www.kurzweiltech.com/aboutray.html) It is basically that exponential technology patterns go beyond Moore’s Law for semiconductor complexity and are applicable to technology in general.

[viii]Ackoff, Russell L., Ackoff’s Best: His Classic Writings on Management, John Wiley & Sons, 1999, pp. 8-24

[ix]McGrath, Michael, E. Product Strategy for High Technology Companies: Accelerating your Business to Web Speed. McGraw-Hill, 2001.

[x]Porter, Michael E., Competitive Strategy: Techniques for Analyzing Industries and Competitors, Free Press, 1998

[xi]Lougee, H., Thurston, G., “System-of-Systems: Development: Competitive Advances in Systems Engineering”, Foliage Whitepaper, 2010 http://www.foliage.com/thought-leadership/whitepapers.php