The Requirements Problem for Adaptive Systems in ACM TMIS

Alex Borgida, Neil Ernst, John Mylopoulos, and I have a new paper out:

Jureta, Ivan J., et al. “The Requirements Problem for Adaptive Systems.” ACM Transactions on Management Information Systems (TMIS) 5.3 (2014): 17.


Requirements Engineering (RE) focuses on eliciting, modeling, and analyzing the requirements and environment of a system-to-be in order to design its specification. The design of the specification, known as the Requirements Problem (RP), is a complex problem-solving task because it involves, for each new system, the discovery and exploration of, and decision making in a new problem space. A system is adaptive if it can detect deviations between its runtime behavior and its requirements, specifically situations where its behavior violates one or more of its requirements. Given such a deviation, an Adaptive System uses feedback mechanisms to analyze these changes and decide, with or without human intervention, how to adjust its behavior as a result. We are interested in defining the Requirements Problem for Adaptive Systems (RPAS). In our case, we are looking for a configurable specification such that whenever requirements fail to be fulfilled, the system can go through a series of adaptations that change its configuration and eventually restore fulfilment of the requirements. From a theoretical perspective, this article formally shows the fundamental differences between standard RE (notably Zave and Jackson [1997]) and RE for Adaptive Systems (see the seminal work by Fickas and Feather [1995], to Letier and van Lamsweerde [2004], and up to Whittle et al. [2010]). The main contribution of this article is to introduce the RPAS as a new RP class that is specific to Adaptive Systems. We relate the RPAS to RE research on the relaxation of requirements, the evaluation of their partial satisfaction, and the monitoring and control of requirements, all topics of particular interest in research on adaptive systems [de Lemos et al. 2013]. From an engineering perspective, we define a proto-framework for solving RPAS, which illustrates features needed in future frameworks for adaptive software systems.

Download PDF:

How is paraconsistent reasoning related to decision-making?

The short answer is that to define a paraconsistent formalism, you have to define decision-making rules. The rules must say what can be concluded from an inconsistent set of formulas. Simply put, your proof theory reflects the decision-making rules that you like, and which you built into the language when you made it.

Even if you are not making the paraconsistent formalism yourself, but, say, are working with a specialist, then you need to define informally, but clearly the decision-making rules first. They should ideally be specific to the problem domain, or problem class that you want to be solving with that formalism.

These are simple observations, and have a crucial implication: there is unlikely to be one best paraconsistent formalism, because there are no universal criteria for which conclusions are valid, when you have an inconsistent set of formulas.

So another important implication is that if you say “I am using paraconsistent formalism X, which someone else made” then you need to make it clear how the decision-making rules in that formalism are good for what you want to use that formalism for. Otherwise, you are picking one, among many, and remains unclear why that one is more relevant than another.

So when making a new, or picking an existing paraconsistent formalism, you need to make it clear why that formalism draws, or better, prefers some conclusions, rather than others, when it is given a set of inconsistent formulas.

What is the minimal number of formulas in a formal logic X, which would be needed to learn the proof theory of X?

Suppose that you have a machine that can produce any number of formulas. You assume that they are all formulas of the same formal logic, called X. How many formulas, and which formulas, would you need, in order to determine all the rules of the proof theory of X?

How my PhD students and I collaboratively write research papers?

In 2014, my research group includes four PhD students, me and another professor.

We apply the following process when collaborating on a research paper:

1) In a Google Drive document, PhD student writes a short and rough motivation and if feasible, something that looks like a research question.

2) PhD student shares the document with me.

3) I add comments and edits via Google Drive.

4) PhD student and I meet in person, or hold a conference call, to discuss the rough motivation and research question.

5) PhD student and I identify relevant existing research, and prioritise it.

6) PhD student revises the Google Drive document, by clarifying the motivation, research question, and related work.

7) PhD student and I meet in person, or hold a conference call, to agree on the research methodology.

8) PhD student and I define the research hypotheses, the required tools for collecting data, or otherwise, as required by the research methodology. These are revised usually in several iterations, until I approve that the student can start applying the tools according to the research methodology.

9) PhD student and I (if I can do something more or different than the student can) collect data, do simulations, and so on, whichever is needed. If we need outside experts to help us, we find them and coordinate with them. We clean data up, and decide if it is worthy of analysis, and of what kind of analysis.

10) PhD student shares dataset on Google Drive, along with any analyses of the data, described in a Google Drive document. PhD student and I separately or together analyse data, decide which results to present, and how to present them.

11) PhD student and I decide on the key ideas and results to present in the research publication.

12) PhD student writes first incomplete draft as a Google Drive document. PhD student and I add comments and edit the document until it is approved as ready for submission. In parallel, we decide on publication venue (specific workshop, conference, journal, book chapter).

13) PhD student transfers the content of the Google Drive draft publication, to LaTex format, if this is required by the publication venue. The LaTex files and the resulting PDF are shared in a folder on Google Drive.

14) I approve the PDF version, and PhD student submits it.

The rest depends on the replies from the reviewers at the publication venue.

The process can vary somewhat, depending on the data to collect, if there is data to collect, problems with data, problems with the hypotheses, or the research question, and so on.

How to design a modelling language ontology from empirical data?

I don’t know how yet. I’ve been thinking about this for about two years now.

The only idea that stuck so far, is to work in five steps.

Firstly, to identify recurring terms in the domain, used to describe problems and solutions.

Then, to somehow estimate their relative importance to people who are experts in that domain, in identifying and solving problems.

Thirdly, to define the concepts for the ontology, and then, the relations. Relations are the harder part.

Finally, to define rules for concept and relation use.

Corentin Burnay and I are going in this direction, in the work on requirements elicitation, but we still have a long way to go. We are, roughly speaking, at the second step.

What is missing in formal models of argument?

Formal models of argument, such as Dung’s argumentation framework, usually do not answer the following interesting questions:
– How to detect groupthink in arguments?
– How to check if arguments are specific enough to the context and topic, so as to sanction the use of generic arguments?
– How to how to detect the availability bias in arguments?
– How to evaluate the relevance of an argument?
– How to evaluate the relevance of the attack of an argument on another one?
– Which extensions to prefer, and why?
– How to detect the manipulation in arguments and attack relation, in favour of one or a subset of extensions?

There are many others, but there is relatively little work that I know of, on the questions above.

How to write emails?

I follow the rules below when writing emails. Many people who work with me do the same. I highly recommend them.

They are inspired by similar rules which Nikola Tosic, Andrea Toniolo, and I designed and use at JTT Partners, to coordinate remote teams efficiently.

If you are a student, apply these rules, and I will reply faster to your email.


– No bcc (blind carbon copy).

– One topic per email.

– Email subject should clearly state the topic.

– One thought per paragraph.

– Short paragraphs.

– One empty line between every two paragraphs.

– One verb per sentence, if feasible. In general: minimise the number of verbs in a single sentence.

– No passive voice.

– No “we”. Say who.

– If you want to ask me something, then include one or more clear questions, which end with the question mark “?”.

– If you want me to do something, then say what, and suggest a deadline.

– If you want to meet me, then propose at least two meeting slots.

– If you need my approval for something, then the word “approval” has to appear in the question.

– Titles and other formalities do not matter to me. I will treat you with the same formalities (or absence therof) that you treat me.

Why a syntactic consequence relation, if it tries to represent human reasoning, is probably not monotonic?

A syntactic consequence relation relates two formulas of a formal logic, if and only if it is possible to apply the proof rules of that logic together with (or on) the first formula, in order to deduce the second formula. The symbol for the syntactic consequence relation is called “turnstile”, is written \vdash in Latex, and looks like “|-“.

So if you write X |- Y in some logic you like, and if the turnstile is well-defined in it, then it means that there is a proof, which can be made using X and the proof theory of that logic, to deduce Y. Note that X may be a set of formulas, which in classical logic is the same as saying that Y is a conjunction of all formulas in the set.

Now, the syntactic consequence relation in classical logic is monotonic, or satisfies the property called monotonicity. This means that if you write X |- Y, and later, you add W to X (say, you make a union of all formulas in W and all those in X), you will also be able to write X + W |- Y, and you would not be wrong. By “+”, I mean some operation where you keep all formulas of X, and add some new ones, which are in W. So you are not removing something from A.

Informally, if you have a syntactic consequence relation, adding new formulas (very roughly, new information) to the set of those you have, still lets you draw (deduce) the same conclusions you could before.

The problem with this, is that it does not look like a good property when you are trying to define a turnstile which somehow resembles human reasoning. The reason is that new information may interact in such ways with old, that you cannot draw the same conclusions. So you still have A+W, but now, you cannot deduce B, or perhaps you can, but at least some of the conclusions that you could draw from A alone, cannot be drawn anymore from A+W.

If a turnstile violates monotonicity, then it is non-monotonic. There is considerable work on non-monotonic logics. I like formal argument systems in particular, and recommend this survey, if you are interested: Chesñevar, Carlos Iván, Ana Gabriela Maguitman, and Ronald Prescott Loui. “Logical models of argument.” ACM Computing Surveys (CSUR) 32.4 (2000): 337-383.

When to solve a decision analysis problem using an argumentation system?

In short, the alternative that maximises expected utility, and is therefore the solution in a decision analysis problem, can be seen as the only acceptable argument in an argumentation system (where you can understand the argumentation system as in, for example Dung, Phan Minh. “On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games.” Artificial intelligence 77.2 (1995): 321-357).

The reason why this simple observation is interesting, is that it is usually hard to elicit or otherwise obtain the information needed to quantify the uncertainty and desirability of alternatives, and from there find one that maximises expected utility.

In contrast, an argumentation system can be constructed by searching for counterarguments for each alternative, until only one is acceptable. The arguments can be any kind of information, as long as the individuals involved in argumentation recognise them as arguments.

In practical terms, when you cannot formulate a problem as a decision analytic one, perhaps you can formulate it as an argumentation system, and look for a solution by adding counterarguments to the system, until only one alternative remains acceptable, and all others are not acceptable.

How to (pragmatically) teach someone the basics of process modelling?

If person A wants to learn the basics of process modelling, have A in a kitchen, have someone else prepare a dish. Ask A to write down, in a numbered list, the steps she would do in order to prepare the same dish herself.

After A does the list, ask her to check the following, and update her list accordingly:
– Is the sequence of steps clear?
– Are inputs to each step clearly stated?
– Are outputs of each step clearly stated?
– Is it clearly stated what to do in each step, to produce outputs from inputs?
– Is it clear who does each step?
– Is it clear what tools and resources (fruits, vegetables, eggs, etc.) are needed at each step?
– Are there steps which need to be repeated? Is this clear from the list?
– Are there steps to do in parallel? Is this clear from the list?
– Does the list say which conditions should be satisfied, in order to start and stop a step (something should have changed colour, texture, water should have come to a boiling point, etc.)?

After they have updated the list, recheck these same questions, and do so until the list of steps is satisfactory.

Then, ask A to make the dish herself, and during and after this, ask her to recheck the same questions, and update her list of steps according to the experience of preparing the dish.

After the above, the person will probably more easily grasp the purpose of process modelling, the difficulties, limitations, and probably process modelling languages as well.

What is the relationship between business analysis and decision-making?

If a decision is a commitment to a course of action, then decision-making designates the cognitive processes that result in such commitments.

The conservative view of decision-making, visible in such fields as expected utility theory in economics and decision analysis in management science, is that the individual who makes the decision identifies alternative actions in a given situation, evaluates the each alternative relative to others, and chooses the one which is, according to some set of criteria which this person adopted, the best among the alternatives. The decision is the commitment to act according to the best alternative. For less conservative views in the said fields, search for “nonstandard utility theory” at Google Scholar, for example.

Business analysis involves decision-making, but has three differences, one in terms of focus, and two in terms of scope. One is that decision-making theories usually are not not specific to a domain, even if it is as broad as “business”, while business analysis focuses on business situations. The second is that decision-making is less concerned than business analysis with methods to apply, to collect and elicit the information about the problem, alternatives, and criteria. The third is that business analysis usually produces advice, rather than the actual commitment, the former being a recommendation on how to act, the latter being the adoption of a course of action.

In short, then, business analysis involves decision-making, but focuses on decisions related to business problems / opportunities, is interested in how to get information for, then design / define the problem, alternatives, and criteria, and produces advice on what to decide, not decisions themselves.

What is a business analysis method?

A business analysis method is a set of well-defined tasks to do, either in order to solve problems which are obstructing organisations to create value, or to realise opportunities that these organisations have identified.

Tasks are well-defined if it is clear:
– why they need to be done,
– what their inputs and outputs are,
– how to transform the inputs into outputs,
– what skills are required to do so,
– which resources are used.

It is at least as important that it be clear:
– why some task has to be done instead of another,
– why it has to be done as described and not in another way, and
– which tasks precede it, follow it, and can or should be done in parallel to it.

A business analysis method is likely to include tasks which explain how to do the following:

– understand the problem or opportunity, which involves preparation, to understand the terminology and practices in the relevant problem domain, and the elicitation of information and knowledge from that problem domain and the people involved;

– synthesis of the collected information and knowledge in order to produce a clear formulation of the problem;

– design of the solutions, which results in the description of alternative solutions;

– evaluation of the alternatives, which consists of identifying their respective merits, limitations, risks;

– selection of one of the alternatives, based on the results of the evaluation;

– recommendation to implement the chosen alternative; and

– supervision of the implementation of the solution, and its adjustment based on what is observed during implementation and use of the solution.

What is the “median entrepreneurial startup”?

It is the average new tiny or small company, rather than the rare, highly successful startups described in the press. It is useful to leave the hype aside, and keep in mind the following, when deciding to invest in, work with, or work for most startups:

“Starting such a firm is like entering a lottery (Storey, 2011; Vivarelli, 2011: 201), with high death rates, skewed returns with most players losing out, random growth, little or no entrepreneurial learning (“learning to roll a dice” [Frankish et al., 2013]), no influence of education on performance, little control over outcomes but substantial overconfidence among players. Like the median lottery player who does not make money after arguably irrationally entering a game where the average payoff is less than the ticket price, most entrepreneurs do not gain a wage premium compared with waged workers. Like lottery players they are psychologically happier, which may be related to them being more optimistic and overconfident (Camerer and Lovallo, 1999; Parker, 2004). As with lottery players, it is not clear that unsuccessful entrepreneurs should be encouraged or subsidized to try again, given that the evidence on entrepreneurial learning from large-scale studies of unsuccessful entrepreneurs is generally weak (Metzger, 2006; Frankish et al., 2013). And lastly, as with lottery players, a tiny minority of “winners” is very visible in the popular press, while the large number of losers is overlooked.”

The quote above is from: Nightingale, Paul, and Alex Coad. “Muppets and gazelles: political and methodological biases in entrepreneurship research.” Industrial and Corporate Change 23.1 (2014): 113-143.

Despite all that, I like working with happy people, so I appreciate working with all kinds of startups. I recommend the same to you :-)

Distinguished paper award at CAiSE 2014

Corentin, Stephane, and I received the distinguished paper award at the 26th International Conference on Advanced Information Systems Engineering, for this paper:

Burnay, Corentin, Ivan J. Jureta, and Stéphane Faulkner. “An Exploratory Study of Topic Importance in Requirements Elicitation Interviews.” Advanced Information Systems Engineering. Springer International Publishing, 2014.

CAiSE selected three papers, of which one was elected the best paper, and two are selected as distinguished papers. Ours was one of the two distinguished papers. Great news :-)

Why is it hard to analyse decision methods?

A decision method explains how an individual, or a group of people make a decision. If a decision method is known, and can be taught, then it can be applied by others, when they need to make decisions.

Democratic elections are an example of a well known decision method. If you want to elect a president of something via democratic elections, then there is a set of rules you need to apply, such as that every vote counts as one (and not more or less than) vote, the candidate who receives more than half of all votes wins, and so on, no voter should be coerced to vote, and so on. Various decision methods that people and animals apply are surveyed in Conradt, Larissa, and Christian List. “Group decisions in humans and animals: a survey.” Philosophical Transactions of the Royal Society B: Biological Sciences 364.1518 (2009): 719-742.

Now, suppose that you need to analyse a previously unknown decision method. By this, I mean that you are given access to the individual or more of them, who are using some rules to make decisions in relation to a particular decision problem, but there is no documentation of these rules, and you want to define the rules that they apply, then see if there are ways to change tem, in order to improve some aspect of their decision making, such as remove some bias, speed it up, etc.

This is hard to do for many reasons. For example, it may be hard for them to give you the rules when asked, because they do not see clearly the regularities in their decision making (or there may be none).

Another issue is that it can be difficult to isolate the actual decision you want to focus on. If a decision is a commitment to a course of action, then it may not be accessible to you, since commitments may only have direct effects on these people’s thinking, and only later produce tangible changes that you can observe (a document which says when and what the person decided, for example, such as a contract).

A third issue is that it may be difficult to isolate a decision problem that you want to analyse the decision method for, because it may be occurring systematically together with some other decision problem.

The issues above, and others, are nicely discussed in Langley, Ann, et al. “Opening up decision making: The view from the black stool.” organization Science 6.3 (1995): 260-279.