|
CENTRAL RESEARCH QUESTION
How to rigorously analyse qualitative information in a decision situation before making a choice? Choice is usually informed: it is on the basis of the information available prior to deciding that one interprets the decision situation, identifies the alternatives and assesses the potential effects thereof. Choices are, however, rarely informed by quantitative information, especially those obtained through rigorous measurement, e.g., as in biology, physics, or chemistry.
Qualitative information is much more readily available to inform choice. Analysis of qualitative information in a decision situation should identify and remove deficiencies, establish how the information may be misleading, among others, as well as inform what additional information to search for and how to do so. I am interested in making, testing, and using conceptual, logical, and methodological means that can be used to analyse qualitative information in order to better inform choice. |
![The Executive Committee of the National Security Council (commonly referred to as simply the Executive Committee or EXCOMM) was a body of United States government officials that convened to advise President John F. Kennedy during the Cuban Missile Crisis in 1962. [source: wikipedia]](http://jureta.net/wp-content/uploads/2010/04/cuba.jpg "The Executive Committee of the National Security Council (commonly referred to as simply the Executive Committee or EXCOMM) was a body of United States government officials that convened to advise President John F. Kennedy during the Cuban Missile Crisis in 1962. [source: wikipedia]")
Above: Meeting of the EXCOMM, a committee formed to advise US President J.F. Kennedy during the Cuban Missile Crisis in 1962. It is a myth that decision making in such situations relies on rigorously acquired quantitative information, such as, e.g., those arising from controlled experiments in physics. There is usually little to no rigorously acquired quantitative information that could be used to inform the decision. Instead, there is advice, diverse agendas, power plays, emotion, constrained communication, all of which give rise to variously deficient, distorted, and misleading qualitative information with which the decision maker must work.
If you are interested in finding out more about the Cuban missile crisis, take a look at the Wikipedia article, you can also listen to the tapes of their meetings, or if you prefer an even more dramatic version, watch the film “13 days” (here’s a trailer). Image credits: Cecil Stoughton, White House, in the John F. Kennedy Presidential Library and Museum, Boston. |
The research project, in more detail
A. Research aims
- Construct, refine, and empirically test novel conceptualizations, mathematical models, and methodologies for decision making in design/engineering decision settings characterized by unavailable/incomplete quantitative estimates of probability and/or utility and the availability of variously imprecise, vague, incomplete, conflicting, and unstable/changing qualitative decision information and advice from potentially non-expert stakeholders.
- Generalize relevant results to non-design/non-engineering decision settings having the said characteristics.
- Grow an interdisciplinary research group focused on this topic at the Louvain School of Management and perhaps elsewhere. If you are interested in cooperating with our group, email me.
- Introduce relevant results into curricula of undergraduate and graduate students at the Louvain School of Management, and potentially other faculties, such as computer science (in relation to software engineering) and/or medicine (in relation to diagnostic procedures). If you are interested in using parts of, or my entire lectures, you can have a look at the lectures section; if you do decide to use my material, it would be nice if you let me know.
B. State of the art
Decision theory, as studied in economics and philosophy [1-4] focuses on what decisions are made rather than how decisions are made, i.e., how is information elicited, analyzed, trans-formed, and used to define the parameters of the decision problem (goals, preferences, and so on), alternative solutions to it (i.e., alternative choices), all of which precede the step which has traditionally been the focus of decision theory: the use of a decision rule to rank solutions so as to single out the most appropriate one according to some criterion. Drawing on individual decision theory, social choice theory, and game theory, decision analysis [5-7] developed as a set of conceptualizations, mathematical models and methodological prescriptions about how individuals should decide. Being an applied decision theory, concepts of probability (for the quantification of uncertainty) and utility (for the quantification of relative desirability) are central to decision analysis. Instead of committing to the demanding objective probability conceptualization [8,9], decision analysis follows the subjective variant [10], relying on Bayesian statistics, while Bayesian belief networks [11] are used in more recent contributions [7] to model interactions and dependencies between (uncertain) sources of information. On utility, decision analysis follows expected utility theory or its variants [4,12,13]. While it is clear in decision analysis that realistic decision makers are not perfectly rational utility maximizers, decision analysis does purport to assist them in approximating to feasible extents such choice behavior.
Qualitative decision theory [14-17] developed in artificial intelligence as a response to practical difficulties of making quantitative probability and utility estimates in many actual decision settings, as well as concerns over the treatment of unstable goals and preferences, procedures for the incremental elicitation of decision information (i.e., how to elicit preferences), and the usual view of decision theory that outcomes are unrefinable (i.e., an outcome is refinable if it can be decomposed so as to search for the utilities and/or probabilities of its “parts” and only then compute its aggregate utility and probability). Interest in qualitative representations of decision information led to the study of mathematical logics that incorporate order relations for preference [18,19]. Relations to expected utility maximization [20] and game theory [21] have also been studied. Discussions of qualitative decision theories and methodologies in management science [22-27] are isolated, rare, and to a considerable extent independent from artificial intelligence research on the topic.
Requirements engineering developed independently from decision theory, decision analysis, and qualitative decision theory, initially (in the 1970s) as a subfield of software engineering [28-31]. The field was originally interested specifically in how to best determine and describe the purpose that software should have within a given human operating environment. As the importance of automated systems to the work and coordination of people increased – and interest shifted from software and hardware alone, to sociotechnical systems – it was recognized that the field must account for the variously precise and (in)consistent expectations of the stakeholders of the system-to-be, including its future users, owners, and so on. An understanding of the functions of the system-to-be could only be sought after the beliefs, desires, and intentions of the stakeholders had been grasped to the feasible extent, leading to (since the 1990s) an interdisciplinary approach which draws from mathematical logics (for the specification and analysis of business processes and organizational structures, and software properties), method engineering, linguistics, artificial intelligence, and knowledge representation and reasoning, among others. Four intertwined questions are central in the field: (i) What information should be elicited from the stakeholders of the system-to-be (whereby the system-to-be is the “organizational system”, i.e., a combination of policies, business processes, automated systems that we are interested in building or changing)? (ii) What models should be used to represent the elicited information? (iii) What kinds of reasoning should be performed over the models of requirements? (iv) How to use the three said components in practice? Seminal answers took the form of requirements modeling languages and methodologies, which typically included (i) an ontology of requirements to state what information to elicit and that is relevant to describe the properties and behaviors of the system-to-be and its operating environment, (ii) modeling primitives corresponding to the concepts and relations of the ontology, the instances of which together form models to capture the elicited information, (iii) variously automated methods applied over the models in order to answer questions of methodological interest, such as whether a model is consistent, or if the properties and behaviors it attributes to the system allow the latter to satisfy its designated purpose, and (iv) guidelines for the use of the said modeling language components. As requirements engineering grew out of engineering concerns, it made no assumptions about the behavior of the organizational stakeholders whose requirements are being elicited, so that the conceptual and methodological contributions of the field rely neither on a variant of ex-pected utility theory nor qualitative decision theory models. There is in fact no “model of the decision maker” in the field. This of course has its disadvantages (e.g., little discussion of the notion of rationality, optimality criteria for decisions, and so on), but the upside is that it developed very concrete approaches – via the modeling languages – to the unavoidable problem of obtaining, analyzing, and structuring information needed to determine the properties of the organizational system that needs to be developed or changed.
Decision theory, as studied in economics and philosophy [1-4] focuses on what decisions are made rather than how decisions are made, i.e., how is information elicited, analyzed, trans-formed, and used to define the parameters of the decision problem (goals, preferences, and so on), alternative solutions to it (i.e., alternative choices), all of which precede the step which has traditionally been the focus of decision theory: the use of a decision rule to rank solutions so as to single out the most appropriate one according to some criterion. Drawing on individual decision theory, social choice theory, and game theory, decision analysis [5-7] developed as a set of conceptualizations, mathematical models and methodological prescriptions about how individuals should decide. Being an applied decision theory, concepts of probability (for the quantification of uncertainty) and utility (for the quantification of relative desirability) are central to decision analysis. Instead of committing to the demanding objective probability conceptualization [8,9], decision analysis follows the subjective variant [10], relying on Bayesian statistics, while Bayesian belief networks [11] are used in more recent contributions [7] to model interactions and dependencies between (uncertain) sources of information. On utility, decision analysis follows expected utility theory or its variants [4,12,13]. While it is clear in decision analysis that realistic decision makers are not perfectly rational utility maximizers, decision analysis does purport to assist them in approximating to feasible extents such choice behavior.
Qualitative decision theory [14-17] developed in artificial intelligence as a response to practical difficulties of making quantitative probability and utility estimates in many actual decision settings, as well as concerns over the treatment of unstable goals and preferences, procedures for the incremental elicitation of decision information (i.e., how to elicit preferences), and the usual view of decision theory that outcomes are unrefinable (i.e., an outcome is refinable if it can be decomposed so as to search for the utilities and/or probabilities of its “parts” and only then compute its aggregate utility and probability). Interest in qualitative representations of decision information led to the study of mathematical logics that incorporate order relations for preference [18,19]. Relations to expected utility maximization [20] and game theory [21] have also been studied. Discussions of qualitative decision theories and methodologies in management science [22-27] are isolated, rare, and to a considerable extent independent from artificial intelligence research on the topic.
Requirements engineering developed independently from decision theory, decision analysis, and qualitative decision theory, initially (in the 1970s) as a subfield of software engineering [28-31]. The field was originally interested specifically in how to best determine and describe the purpose that software should have within a given human operating environment. As the importance of automated systems to the work and coordination of people increased – and interest shifted from software and hardware alone, to sociotechnical systems – it was recognized that the field must account for the variously precise and (in)consistent expectations of the stakeholders of the system-to-be, including its future users, owners, and so on. An understanding of the functions of the system-to-be could only be sought after the beliefs, desires, and intentions of the stakeholders had been grasped to the feasible extent, leading to (since the 1990s) an interdisciplinary approach which draws from mathematical logics (for the specification and analysis of business processes and organizational structures, and software properties), method engineering, linguistics, artificial intelligence, and knowledge representation and reasoning, among others. Four intertwined questions are central in the field: (i) What information should be elicited from the stakeholders of the system-to-be (whereby the system-to-be is the “organizational system”, i.e., a combination of policies, business processes, automated systems that we are interested in building or changing)? (ii) What models should be used to represent the elicited information? (iii) What kinds of reasoning should be performed over the models of requirements? (iv) How to use the three said components in practice? Seminal answers took the form of requirements modeling languages and methodologies, which typically included (i) an ontology of requirements to state what information to elicit and that is relevant to describe the properties and behaviors of the system-to-be and its operating environment, (ii) modeling primitives corresponding to the concepts and relations of the ontology, the instances of which together form models to capture the elicited information, (iii) variously automated methods applied over the models in order to answer questions of methodological interest, such as whether a model is consistent, or if the properties and behaviors it attributes to the system allow the latter to satisfy its designated purpose, and (iv) guidelines for the use of the said modeling language components. As requirements engineering grew out of engineering concerns, it made no assumptions about the behavior of the organizational stakeholders whose requirements are being elicited, so that the conceptual and methodological contributions of the field rely neither on a variant of ex-pected utility theory nor qualitative decision theory models. There is in fact no “model of the decision maker” in the field. This of course has its disadvantages (e.g., little discussion of the notion of rationality, optimality criteria for decisions, and so on), but the upside is that it developed very concrete approaches – via the modeling languages – to the unavoidable problem of obtaining, analyzing, and structuring information needed to determine the properties of the organizational system that needs to be developed or changed.
C. Research project
Fundamental critiques of the relevance of expected utility theory as of a realistic model of a rational decision maker are well known [32-34]. Decision analysis, as a methodological compa-nion to decision theory, remains applicable in decision settings where its assumptions are satisfied. Qualitative decision theory reflects the inadequacies of classical decision theory and decision analysis in many realistic settings, in which decision analytic assumptions do not apply. However, qualitative decision theory has not been studied as an alternative to classical decision theory in management science research and education: what is thus, roughly speaking, missing in management are conceptualizations, mathematical models, and methodologies which would together form an applied counterpart to qualitative decision theory thus being what decision analysis was/is for classical decision theory.
It is uncontroversial that in very many decision settings, the decision maker has variously imprecise, vague, incomplete, conflicting, and unstable/changing qualitative decision informa-tion and advice from potentially non-expert stakeholders. It is for such settings that qualitative decision theory aims to formulate decision rules and algorithms for the identification of alterna-tives that satisfy decision rules. An applied qualitative decision theory – one that should stand side by side with decision analysis in management science – requires that we answer a different and complementary set of questions to those addressed by qualitative decision theory, namely: what are the concepts, the informal and the mathematical models that instantiate and relate instances of these concepts, what are the analyses to apply to models, and how should the concepts, models, and analyses be used in practice in order to produce, as their output in a given actual decision setting, the definition of the decision problem and of the alternative solutions to it, so that the decision rules and algorithms of qualitative decision theory can be applied in order to single out the solution that satisfies a selected decision rule. Stated otherwise, qualitative decision analysis should guide the search for and transformation of the decision information that are necessary for the application of results from qualitative decision theory.
I recognized during my PhD studies that the conceptual and methodological questions which should be answered in order to introduce qualitative decision analysis are rigorously studied not in established decision analysis, but in requirements engineering: the four questions (i)-(iv) mentioned above as central to the latter field correspond to the concerns that any form of qualitative decision analysis should address. In other words, construction of a qualitative deci-sion analysis should select from and build on results from requirements engineering. This was not possible, however, since it was unclear in what ways the longstanding core conceptualiza-tion and formulation of the requirements problem [35] relates to the conceptualization (via concepts of preference, alternative, and so on) of the decision problem in classical or qualitative decision theory. Our redefinition of the requirements problem and of the core ontology for requirements engineering [36] addressed this issue by reformulating the requirements problem as a decision problem in qualitative decision theory, opening thereby the possibility for the successful realization of this research project. The primary aim of this project is thus to start from this result and provide conceptualizations, informal and mathematical models, analysis techniques, and methodologies (by drawing in part from my prior work requirements engineering) then empirically test these to form a qualitative decision analysis. In more specific terms, and given that the contributions are anchored in my prior work, this means that a first qualitative decision analysis I will be proposing shall use not quantitative estimates for utility and probability, but models decision information and the problem as theories of a non-monotonic and paraconsistent propositional mathematical framework with order relations to reflect relative desirability and uncertainty (e.g., [37] for a sample of my ongoing work on this topic), and shall incorporate advanced conceptual analysis (see [38] for a sample of my ongoing work on the use of conceptual analysis in various decision settings). Connection of this work to decision rules and algorithms of qualitative decision theory will be done. Since my colleagues and me focused on decision settings in information systems engineering, this first variant of qualitative decision analysis will be specialized to such application areas. Hence the second aim of this project, namely to determine modifications to that first set of research results that should be made in order to make our qualitative decision analysis general, i.e., applicable to, e.g., strategy definition. This will also involve establishing how our qualitative decision analysis should address decision settings in which some quantitative estimates of probability are available, and hence the link to established decision analysis. Given how decision analysis evolved since its introduction in the 1960s (first a Stanford research group, continual research, applications in industry and policy making), I am convinced that there is potential for successful realization of the third aim of this project: the creation of a research group dedicated to the elaboration, testing and specialization of anticipated results to various areas of management science, and hopefully other fields in which professionals encounter decision settings having the characteristics described earlier. Given that many decision settings make established decision analysis inapplicable, the research results will be reformulated in a format that allows the creation of new lectures for curricula in management science at the Louvain School of Management, and potentially other faculties: requirements engineering is taught also in computer science, and our work will bring advances in decision making to state of the art requirements engineering. Initial work towards this aim is ongoing, as Prof. Stéphane Faulkner and I already introduced a new lecture that emphasizes conceptual analysis in managerial decision making, based in a significant part on selected chapters from my manuscript on the analysis of advice [39]. Successful completion of this project will significantly advance in conceptual, mathematical and methodological terms the available, but rare and isolated contributions that historically precede ours in management science [22-27].
Fundamental critiques of the relevance of expected utility theory as of a realistic model of a rational decision maker are well known [32-34]. Decision analysis, as a methodological compa-nion to decision theory, remains applicable in decision settings where its assumptions are satisfied. Qualitative decision theory reflects the inadequacies of classical decision theory and decision analysis in many realistic settings, in which decision analytic assumptions do not apply. However, qualitative decision theory has not been studied as an alternative to classical decision theory in management science research and education: what is thus, roughly speaking, missing in management are conceptualizations, mathematical models, and methodologies which would together form an applied counterpart to qualitative decision theory thus being what decision analysis was/is for classical decision theory.
It is uncontroversial that in very many decision settings, the decision maker has variously imprecise, vague, incomplete, conflicting, and unstable/changing qualitative decision informa-tion and advice from potentially non-expert stakeholders. It is for such settings that qualitative decision theory aims to formulate decision rules and algorithms for the identification of alterna-tives that satisfy decision rules. An applied qualitative decision theory – one that should stand side by side with decision analysis in management science – requires that we answer a different and complementary set of questions to those addressed by qualitative decision theory, namely: what are the concepts, the informal and the mathematical models that instantiate and relate instances of these concepts, what are the analyses to apply to models, and how should the concepts, models, and analyses be used in practice in order to produce, as their output in a given actual decision setting, the definition of the decision problem and of the alternative solutions to it, so that the decision rules and algorithms of qualitative decision theory can be applied in order to single out the solution that satisfies a selected decision rule. Stated otherwise, qualitative decision analysis should guide the search for and transformation of the decision information that are necessary for the application of results from qualitative decision theory.
I recognized during my PhD studies that the conceptual and methodological questions which should be answered in order to introduce qualitative decision analysis are rigorously studied not in established decision analysis, but in requirements engineering: the four questions (i)-(iv) mentioned above as central to the latter field correspond to the concerns that any form of qualitative decision analysis should address. In other words, construction of a qualitative deci-sion analysis should select from and build on results from requirements engineering. This was not possible, however, since it was unclear in what ways the longstanding core conceptualiza-tion and formulation of the requirements problem [35] relates to the conceptualization (via concepts of preference, alternative, and so on) of the decision problem in classical or qualitative decision theory. Our redefinition of the requirements problem and of the core ontology for requirements engineering [36] addressed this issue by reformulating the requirements problem as a decision problem in qualitative decision theory, opening thereby the possibility for the successful realization of this research project. The primary aim of this project is thus to start from this result and provide conceptualizations, informal and mathematical models, analysis techniques, and methodologies (by drawing in part from my prior work requirements engineering) then empirically test these to form a qualitative decision analysis. In more specific terms, and given that the contributions are anchored in my prior work, this means that a first qualitative decision analysis I will be proposing shall use not quantitative estimates for utility and probability, but models decision information and the problem as theories of a non-monotonic and paraconsistent propositional mathematical framework with order relations to reflect relative desirability and uncertainty (e.g., [37] for a sample of my ongoing work on this topic), and shall incorporate advanced conceptual analysis (see [38] for a sample of my ongoing work on the use of conceptual analysis in various decision settings). Connection of this work to decision rules and algorithms of qualitative decision theory will be done. Since my colleagues and me focused on decision settings in information systems engineering, this first variant of qualitative decision analysis will be specialized to such application areas. Hence the second aim of this project, namely to determine modifications to that first set of research results that should be made in order to make our qualitative decision analysis general, i.e., applicable to, e.g., strategy definition. This will also involve establishing how our qualitative decision analysis should address decision settings in which some quantitative estimates of probability are available, and hence the link to established decision analysis. Given how decision analysis evolved since its introduction in the 1960s (first a Stanford research group, continual research, applications in industry and policy making), I am convinced that there is potential for successful realization of the third aim of this project: the creation of a research group dedicated to the elaboration, testing and specialization of anticipated results to various areas of management science, and hopefully other fields in which professionals encounter decision settings having the characteristics described earlier. Given that many decision settings make established decision analysis inapplicable, the research results will be reformulated in a format that allows the creation of new lectures for curricula in management science at the Louvain School of Management, and potentially other faculties: requirements engineering is taught also in computer science, and our work will bring advances in decision making to state of the art requirements engineering. Initial work towards this aim is ongoing, as Prof. Stéphane Faulkner and I already introduced a new lecture that emphasizes conceptual analysis in managerial decision making, based in a significant part on selected chapters from my manuscript on the analysis of advice [39]. Successful completion of this project will significantly advance in conceptual, mathematical and methodological terms the available, but rare and isolated contributions that historically precede ours in management science [22-27].
D. References
- R. C. Jeffrey. The Logic of Decision. Univ. Chicago Press, 1990.
- M. D. Resnik. Choices: An Introduction to Decision Theory. Univ. Minnesota Press, 1987.
- J. M. Joyce. The Foundations of Causal Decision Theory. Cambridge Univ. Press, 1999.
- J. Von Neumann, O. Morgenstern. Theory of Games and Economic Behavior. Princeton Univ. Press, 1956.
- R. A. Howard. Decision analysis: Applied decision theory. Proc. 4th Int. Conf. Operational Research, 1966.
- R. L. Keeney, H. Raïffa. Decisions with multiple objectives: preferences and value tradeoffs. Wiley, 1976.
- W. Edwards, R. F. Miles, D. Von Winterfeldt (Eds.). Advances in decision analysis: from foundations to applications. Cambridge Univ. Press, 2007.
- L. E. Szabó. What remains of probability? In D. Dieks, W. Gonzalez, S. Hartmann, M. Weber, F. Stadler, T. Uebel (Eds.), The Present Situation in the Philosophy of Science. Springer, 2009.
- Glenn Shafer. From Cournot’s Principle to Market Efficiency. Tech. Report, Rutgers Univ., 2006.
- B. de Finetti. Theory of Probability. Wiley, 1974.
- J. Pearl. Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, 1988.
- P. J. H. Schoemaker. The expected utility model. J. Economic Literature, 20(2):529-563, 1982.
- C. Starmer. Developments in non-expected utility theory: The hunt for a descriptive theory of choice under risk. J. Economic Literature, 38(2):332-382, 2000.
- J. Pearl. From conditional oughts to qualitative decision theory. In Conf. Uncertainty in Artif. Intel., 1993.
- R. Brafman, M. Tennenholtz. On the axiomatization of qualitative decision theory. In Conf. Artif. Intel., 1997.
- D. Dubois, H. Prade, R. Sabbadin. Decision-theoretic foundations of qualitative possibility theory. Eur. J. Op. Res. 128:459-478, 2001.
- J. Doyle, R. H. Thomason. Background to Qualitative Decision Theory. AI Magazine, 20(2) 55-68, 2000.
- S. Hanson. Preference-based Deontic Logic. J. Phil. Logic 19:75-93, 1990.
- D. Dubois, H. Fargier, P. Perny. Qualitative decision theory with preference relations and comparative uncertainty. Artificial Intel. 148:219-260, 2003.
- H. Fargier, R. Sabbadin. Qualitative decision under uncertainty. Artificial Intell. 164(1-2):245-280, 2005.
- P. M. Dung. On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games. Artificial Intell. 77:321-357, 1995.
- I. I. Mitroff, F. Betz. Dialectical Decision Theory. Management Sci. 19(1):11-24, 1972.
- R. O. Mason. A Dialectical Approach to Strategic Planning. Management Sci. 15(8): B403-B414, 1969.
- I. I. Mitroff, R. O. Mason, V. P. Barabba. Policy as Argument. Management Sci., 28(12):1391-1404, 1982.
- C. M. Brugha. The structure of qualitative decision-making. Eur. J. Op. Res. 104:46-62, 1998.
- A. v. Werder. Argumentation Rationality of Management Decisions. Org. Sci. 10(5):672-690, 1999.
- A. J. Feelders, H. A. M. Daniels. A general model of automated business diagnosis. Eur. J. Op. Res. 130:623-637, 2001.
- I. Sommerville, P. Sawyer. Requirements Engineering: A Good Practice Guide. John Wiley & Sons, 1997.
- P. Loucopoulos, V. Karakostas. System Requirements Engineering. McGraw-Hill, 1995.
- K. Pohl. Process-centered requirements engineering. John Wiley & Sons, 1996.
- M. Jirotka, J. A. Goguen (Eds.). Requirements engineering: social and technical issues. Acad. Press, 1994.
- M. Allais. Le comportement de l’homme rationnel devant le risque. Econometrica 21:503-546, 1953.
- D. Ellsberg. Risk, ambiguity and the Savage axiom. Quarterly J. Econom. 75:643-669, 1961.
- D. Kahneman, A. Tversky. Prospect theory and analysis of decision under risk. Econometrica 47, 1979.
- P. Zave, M. Jackson. Four Dark Corners of Req. Eng. ACM Trans. Softw. Eng. Methodol. 6(1):1-30, 1997.
- I. Jureta, J. Mylopoulos, S. Faulkner. A core ontology for requirements. Applied Ontology 4(3-4):169-244, 2009.
- I. Jureta, A. Borgida, J. Mylopoulos, N. Ernst. Techne. Tech. Rep. CSRG-606, University of Toronto, 2010. Available online.
- I. Jureta. Analysis of Advice. Manuscript in progress.
- I. Jureta, S. Faulkner. EIMI-B311: Decision Making & Requirements Engineering. Lecture at the Louvain School of Management – FUNDP, held for the first time to undergraduate students in Sept.-Dec. 2009.