Is requirements engineering a special case of decision analysis?

Voiced by Amazon Polly

“Decision analysis” can be used to mean any activity that tries to find reasons for a decision, but in research, it usually denotes an approach that was proposed, to analyze decisions before making them.

Drawing on utility theory in particular, “decision analysis” was introduced, as far as I know by Ron Howard [1] at the end of the 1960s, with the aim to approach decision-making in a rigorous way, while trying to reuse ideas from expected utility theory in economics. Here is how he argued for the need for decision analysis, and how he defined it.

“A decision is an irrevocable allocation of resources, irrevocable in the sense that it is impossible or extremely costly to change back to the situation that existed before making the decision. […] a decision is not a mental commitment to follow a course of action but rather the actual pursuit of that course of action. […]

A good decision is a logical decision – one based on the uncertainties, values and preferences of the decision maker.  A good outcome is one that is profitable or otherwise highly valued. In short, a good outcome is one we wish would happen. […] We may be disappointed to find that a good decision has produced a bad outcome or dismayed to learn that someone who has made what we consider to be a bad decision has enjoyed a good outcome. Yet, pending the invention of the true clairvoyant, we find no better alternative in the pursuit of good outcomes than to make good decisions.”

The aim with decision analysis, in other words, is to find ways of making “good decisions”, and hope that good outcomes will follow. This is sometimes called “procedural quality” of decision-making, which evaluates the process we took to make a decision, while “outcome quality” is concerned with the evaluation of outcomes. What Howard is saying is simply that we want procedures that aim for high procedural quality, since we do not see another way to influence outcome quality.

“Decision analysis is a logical procedure for the balancing of the factors that influence a decision. The procedure incorporates uncertainties, values, and preferences in a basic structure that models the decision. […] The essence of the procedure is the construction of a […] model of the decision in a form suitable for computation and manipulation; the realization of this model is often a set of computer programs.”

The motivation for decision analysis looks a lot like those that led to the research in requirements engineering: we want to have rigorous approaches to finding, representing, analyzing, and specifying requirements. If we don’t, we will specify wrong and unclear requirements; systems will be built to satisfy these deficient requirements, they will fail to deliver what stakeholders expect of them, and eventually, their design, engineering, and running will be, to a considerable extent, waste. To fail requirements engineering is to fail to relate expectations from a system to the purpose for which it is being designed.

So how are decision analysis and requirements engineering related?

When you do requirements engineering for a system, you make many decisions: how to elicit requirements, how to model them, how to analyze them, which conclusions to draw from the analysis, what to do about these conclusions, how to verify requirements consistency, among others, how to validate requirements with stakeholders to make sure you got them right – and these questions are only about how you do requirements engineering; there are many decisions to make about the actual content, the information you work with during requirements engineering: given some input from stakeholders, do you make it into a requirement, a constraint of the environment in which the system will run, do you refine it, do you look for more information to relate to it, and so on.

One way to see the connection between decision analysis (DA) and requirements engineering (RE) is that you could approach each decision during requirements engineering in the way that DA tells you to.

While this may be possible, does it make sense to do it?

The decision analysis process is as follows [2]:

Given that we often have hundreds of requirements to deal with, which came from more even more information collected through elicitation, the frequent refinements, clarifications, and changes we have to make to requirements, and the many other choices to make during RE, it is hard to take seriously the idea that decision analysis ought to be applied each time a decision needs to be made during RE.

If this is not the relationship, then what is?

We could, instead, see the requirements engineering process as a special case of decision analysis: the decision to make is which system to design, that is, the purpose of the system-to-be.

But there are problems with this as well. Decision analysis makes a number of assumptions, namely:

  1. The best option is the option which gives the highest expected utility.
  2. It is possible to find or make more than one option to the sufficient level of detail that they can be compared over criteria.
  3. It is possible to operationalize goals into criteria.
  4. Preference and importance can capture emotions, moods, expectations.
  5. It is possible to produce a total order of preference, on each criterion, over consequences of options.
  6. It is possible to produce probability estimates on each criterion, for consequences of options.
  7. It is possible to separate the search and design of the best option from the selection of Criteria.

If your way of doing RE satisfies these, then perhaps your requirements engineering process looks like a variant of decision analysis. But these are demanding assumptions, and there is no research that can lead us to conclude it is a process that generates the best results.

References and notes
  1. Howard, Ronald A. “The foundations of decision analysis.” IEEE transactions on systems science and cybernetics 4.3 (1968): 211-219.
  2. Keeney, Ralph L. “Decision analysis: an overview.” Operations research 30.5 (1982): 803-838.
  3. Keeney, Ralph L., and Howard Raiffa. Decisions with multiple objectives: preferences and value trade-offs. Cambridge university press, 1993.