Requirements Elicitation from Multiple Sources: Case Studies

1. Introduction

This paper gives an overview of a requirements elicitation method, referred to as the Source Control Method. The Method’s concepts are demonstrated with two case studies. The cases studies were designed to illustrate different aspects of the Method and to determine its feasibility and its practical utility as a problem investigation and information validation technique. The cases studies collectively address the issues of information evaluation and conflicts resolution in the information coming from the same source or from different sources.

The Viewpoint Resolution Method is a collection of domain-independent heuristics to build internal models of the viewpoints that record their performance in providing information, to assess information, and to resolve conflicts between viewpoints. The starting point in devising such model is to regard requirements elicitation as a belief formation process in the context of a truth maintenance system. The model is designed to operate within an environment that supports relevant aspects of natural language processing.

Figure 1 depicts information evaluation as a belief formation and truth maintenance process.

Figure 2 shows the architecture of underlying model the Source Control Method which guides and coordinates the different processes of the method. The method's processes are driven by a set of domain-independent heuristics to assist an analyst in deciding what to do next, that is, deciding what type of analysis heuristics need to be activated at any point of the investigation. The guide needs to pass on cases, events and viewpoint models from one process to the next so each process has the necessary data to work on and can use the results of previous evaluations in its analysis.

2. An Overview of the Method

Figure 3 shows a SADT [14] model of the method (the figure shows the life histories of an event and a viewpoint model). The Viewpoint Control Method comprises the following activities:

• Universe of Discourse Initialization

○

Information Validation

○

Importance Analysis

○

Information Evaluation

• Communication

○

Conflict analysis

○

Enquiry

○

Universe of Discourse Update

Figure 3 is further decomposed in Figure 4 and Figure 5. The Viewpoint Control activities are driven by a collection of domain-independent heuristics to decide whether or not to take interest in a particular piece of information, in assessing information, in resolving conflicts between pieces of information, in enquiring to produce further information and also in re-evaluating the corresponding viewpoint models.

3. The Case Studies

The key criteria for selecting the case studies are the different kinds of uncertainty that are involved, namely:

• a changing environment/context,
• different people (different skills, goals, commitments),
• many sources of information, and
• various constraints, such as time, money, etc.

The first case study illustrates the investigative nature of the method and the effectiveness of its heuristics in pointing out problems and in detecting pointers to missing information. The second case study concentrates on the communication part of the method: to support group decision under uncertainty. In particular, the study shows how the information produced by the method can help the negotiation of compromises between competing views.

3.1. Case Study 1

3.1.1. Has the Manager a Business Case?

An engineering configuration manager in a start-up company decides to purchase a new word processor for his secretary claiming that it will improve the quality of the control construction. From the secretary's point of view the word processor will save time. The item's cost is estimated at a year's salary for a secretary. The problem for the financial department is to establish whether the manager has a business case or merely wants a new toy and prestige for his secretary. Time and resources constraints are taken into account.

3.1.2. Initial Universe of Discourse

From the problem statement there are four important viewpoints: the manager, the secretary, the financial department, and the analyst. Using the available information, the analyst first creates initial viewpoint models for the manager and the secretary:

The manager’s viewpoint model looks like:

The secretary’s viewpoint looks like:

3.1.3. Importance Analysis

Given the high cost of the item ordered and the position of the manager the analyst has to justify both to the financial department and to the manager the acceptance or rejection of the order. This means any piece of information related to the business situation will be considered as important.

3.1.4. Information Evaluation

The analyst has to evaluate the manager's statement that a word processor improves the quality of his department services. This can be represented by LOLITA (Large-scale Object-based Linguistic Interactor. Translator and Analyser) [13] as the event:

* event1 *

universal: event - 7688 - rank: universal definition,

subject.: processor - 29069 - rank: individual - suspended,

action: improve - 16916 -

object: qualify - 29071 - rank: individual - suspended,

time: present. - 20989 -

date: 26 September 2023

viewpoint: manager - 19845 - rank: named individual

status: suspended. - 29025 –

The analyst finds out that although the manager has good managerial expertise, including budget management, he lacks the appreciation of some of the new technology. The analyst would assign a low belief to the information according the following heuristics:

if the information requires expertise

and the conviction is high (1)

and the ability in that subject is low

then problem of ability→ yes

if the conviction in the information is high

and there is a problem of ability (2)

then belief→ low

Suppose that the analyst decides to consider the problem from the secretary's viewpoint. The analyst has to assess the fact that the word processor will save the secretary's time by first consulting the secretary's model.

Similarly, the analyst concludes that the secretary is behaving irrationally (i.e., reasoning=low) as he is trying to assume technical expertise in the word processing field.

The analyst may choose, instead, to evaluate the information from the secretary against the existing information. In this case the secretary's statement that the word processor saves time is a reiteration of the manager's statement that the word processor improves the quality of the department's services, assuming that saving time is an improvement.

Assuming that the convictions for both statements are relatively high, the analyst gets to the same conclusion (attach low belief to the secretary's statement) using the following heuristic and the results of heuristics (1) and (2):

if the old conviction is equal to the new

and the conviction in the new information is high

and there is no problem of trust (3)

and there is a problem of ability,

then check out ability problem

Suppose that the analyst decides to gain a first-hand experience of how much typing the secretary does in a typical working day. He notices that the secretary spends only one hour and a half typing and spends the rest of the day doing other administrative work (note that event3 below is an example and not an actual output of LOLITA):

Event3:

subject: secretary

action: type

status: real

time: present

viewpoint: analyst

certainty: high (real)

Let's assume that the information from the analyst is credible and does not need further analysis. At this stage the analyst detects a contradiction between the analyst's statement (event3) and the secretary's statement. The analyst could decide to suspend event2 and investigate because event3 is stronger (based on first hand experience) and because of the following heuristic:

if there is a contradiction (4)

and the new information is stronger than the old,

then suspend and investigate

One may argue that event3 weakens event2. Thus, assuming that there are advantages on the part of the secretary in trying to convince the analyst, the result is the need for an enquiry according the following heuristics:

if a piece of information weakens another then the analyst has to check the relative weight of viewpoints and whether there are interests and advantages involved if they are in the same context:

if there are interests and advantages then the analyst will be inclined to start an enquiry unless that is not possible. In the enquiry process independent viewpoints should take precedence.

3.1.5. Enquiry

Suppose that the analyst decides to establish a profile of the secretaries around the whole company. The analyst discovers the existence of a design services pool (DSP). The analyst finds out from its supervisor that the DSP does the typing (seven hours a day) for all the departments, including the engineering configuration department. The analyst is interested in establishing whether the introduction of a word processor to the DSP would improve its productivity.

The analyst can establish this simply by using a mathematical model from operational research (e.g., queuing theory) as a way of checking if the introduction of a word processor cuts down the waiting list for the DSP. Suppose that the analyst concludes that a word processor does cut down the waiting list for the DSP.

The information from the DSP manager weakens the department manager's claim. This leads to further reduction in the credibility of that claim:

if the new information is stronger than the old, and there is no problem of trust or ability (of the DSP manager), then reduce belief (department manager) and add the new viewpoint.

3.1.6. Conflict Resolution

The obvious decision is to reject the department manager's claim because it is incorrect (very low belief assigned). However, this is not the case in practice, according to the general heuristics:

The department manager can not be ignored.

A solution that leaves any group totally unsatisfied can not be accepted.

Elements from each view have to be contained in the proposed solution.

A qualified majority is required.

Thus, there is a need for find a compromise through negotiation between the manager, the financial department and the DSP as well as the analyst. The analyst may propose that the word processor should be purchased for the DSP with one redundancy from that group.

The Viewpoint Control Method does not deal with the negotiation process but the information recorded in the viewpoint models and the cases is meant to be used as part of an agenda for the negotiation. This is illustrated by the second case study.

3.1.7. Universe of Discourse Update

In this case the DSP supervisor viewpoint is added to the universe of discourse together with an initial viewpoint model. Then the manager viewpoint model needs to be updated in the light of the new evidence from the enquiry. A possible parameter to be updated in the manager's model is the 'goals' parameter. Initially, the manager's goal was set to: 'improve the quality of the control construction'. Following the investigation, the goal could be changed to 'gain prestige'. Other defaults values in the viewpoint models could also be confirmed or modified.

The results of the analyses can be recorded in the following case:

case10:

Event: event1

Viewpoint: manager

Importance Analysis

Viewpoint: yes

Information: yes

Information Evaluation

Determination: ok

Problem of Commitment: no

Problem of Advantage: yes

Problem of Ability : yes

Problem of Trust : no

Result: low belief

Conflict Analysis:

Event: event2

Viewpoint: secretary

Same Context: yes

Type: reiteration

Problems of Trust: none

Problems of Ability: yes

Result: keep belief at present level

Viewpoint Re-evaluation

Goals: ??

Expertise: low

Reasoning: high

Judging information: high

Experience: high

Trustworthiness: high

Helpfulness: average

Result: gain prestige

3.2. Case Study 2

3.2.1. Route Generation and Selection

The term 'route' is used in manufacturing industry to mean the design and production phases [15]. The phases are strongly linked since the characteristics of a design determine the manufacturing processes needed and features of the production cycle act as constraints on the acceptable designs. The traditional approach to this problem is sequential and is inadequate to cope profitably with the feedback nature of these processes, which are naturally iterative and interactive.

Thus, to reduce lead time and costs and to improve communication it is necessary to adopt a Simultaneous Engineering approach that allows designers and production engineers to work in parallel and synchronously.

3.2.2. Universe of Discourse Initialization

The initial list of viewpoints is made up of the Simultaneous Engineering Team (SET),

SET = (V_I, V₂, ... , V_k)

For example:

V_I = group of production engineers

V₂ = designer

V₃ = accountants

V₄ = personnel manager

V₅ = marketing manager

etc.

Given a common task (e.g., marketing) of each group it is possible to construct an aggregate

viewpoint model (ability, trustworthiness, goals, etc) for each group using information that can be

obtained from official documents, interviews with members of the group and with external people,

e.g. customers, personal experiences, etc. For the sake of simplicity, we assume that the ability

and trustworthiness of a group can be represented as a single weight. Thus, an initial universe of

discourse can be represented as:

UoD = { <V_l.w₁.g₁>, < V₂.w₂.g₂>. ... < V_k.w_k.g_k> }

w₁, w₂,..., w_k weights assigned to the viewpoints,

g₁, g₂,..., g₃ goals

3.2.3. Importance Analysis

Importance analysis for this case study is roughly identical to the feasibility analysis of the solutions (i.e., production routes). To check the routes feasibility, each group proposes one or more solutions as 'production routes'. It is assumed that each group has evaluated their proposed solutions for their advantages and disadvantages.

Because of the complexity of the problem there may be a number of possible solutions. The number of possible solutions needs to be reduced. Some of these solutions would be unfeasible on the basis of the overall limitation on technical grounds (e.g., if there is a deadline that all solutions have to respect). These solutions have to be eliminated. It should be clarified to the viewpoints why some solutions were declared unfeasible.

Other solutions could be 'similar', according to an ad hoc metric that must be agreed with the customer or the experts before the process starts. In order to reduce the number of solutions further the remaining routes could be classified according to certain properties such as structure (S) and criterion (C). For example S = (machines selected, operator, sequence), C = (cost, relationalstress , quality of the final product, marketability). The solutions can be classified as:

• similar structure and similar goals
• similar structure and different goals
• different structure and similar goals

The result of importance analysis is the set of solutions deemed feasible.

3.2.4. Information Evaluation

Information evaluation estimates the acceptability of each feasible solution. Each production route is evaluated and assigned a belief (weight) - a global 'index of goodness' - in relation to their consistency with the technical, domain dependent knowledge and by checking the internal consistency of each pair route/experts according to the domain independent heuristics (e.g. responsibility vs. advantage).

3.2.5. Universe of Discourse Update

Once the routes have been received the groups' weights are modified using some information about the solutions. For example:

If all of a group's solutions have been eliminated or have received a low index, the weight should be decremented.

If the solutions proposed by young experts are similar to those of senior engineers, their weights should be incremented.

If a solution is such that it maximizes only the goals specific to the group's class, and low commitment is accepted, the weight should be decremented.

The results of this and the previous phases are recorded as cases, with a case for each acceptable solution. For example:

Case20:

Route:_

Structure:_

Criterion:_

Weight:_

Viewpoint:_

Information Evaluation

problem of Commitment:_

Problem of Advantage:_

Problem of Ability:_

Viewpoint Re-evaluation

Weight:_

Goal:_

3.2.6. Group Decision

The next step is concerned with the group decisional process. The process takes as input the set of cases and viewpoint models and selects a compromise. The Consensus Model [17] can be used at this stage:

• Each member of SET is asked to give their evaluation of the proposed solutions with respect to each criterion (i.e. with respect to the satisfaction of their private goal). The results of this phase of consultation is expressed in a matrix form. The elements of the matrix represent the 'linguistic performance' that the group have attributed to each solution with respect to each criterion, (that is, choosing a linguistic label represented by 'fuzzy numbers' in a term set V, the range of which is pre-defined. For example: V = (very low, low, medium, high, very tight),
• Taking into account the weights of the viewpoints, recorded in the viewpoint models, a consensus strategy is identified using a 'cost for changing opinion' (some form of commitment). There are two stages:

• identify candidate solutions for the discussion (eliminate those whose total value - as judged by the viewpoints - does not pass a fixed threshold),
• evaluate the remaining solutions again after a discussion based on the advice of the consensus strategy, which in turn is based on the 'cost for changing opinion': this process may be repeated a number of times depending on various elements. The weights assigned to the viewpoints can also be changed as a result.

• Change the weights of the viewpoints.

If after a round of consultation there is not a Production Route that can acceptable as the final one, it is necessary to change the level of negotiation and change the Production Routes themselves. The viewpoints have to be told why a consensus has not been reached and on how to use this information as a starting point for the next stages.

Note that this example is only intended to show how a negotiation strategy can be, easily, integrated with the techniques of the Viewpoint Control Technique This is due to the flexibility of the method and the usefulness of the information it produces for the process of negotiation

4. Summary and Conclusions

Using two case studies, this paper has demonstrated the utility of the Viewpoint Control Technique in the very early stages of requirements elicitation. Because the Viewpoint Control Method records who said what, the track-record of each source, and what evidence is provided to endorse what was said, it is able to use that evidence to judge the degree of correctness of individual pieces of information. The case studies sought to illustrate the following aspects of the proposed approach:

• the coordination of the different analysis techniques via the viewpoint models and the cases, thus the production of an all-important learning feed-back,
• the flexibility of the technique: the technique can be adapted to different situations in different domains. No order is imposed on performing the analysis tasks, e.g., iteration between conflict resolution and viewpoint re-evaluation, and no specific representation is imposed for expressing the facts,
• the important role human factors play even in technical decisions,
• the ability to detect inconsistencies, wrong information, and incompleteness as the requirements evolve,
• the exploitation of the correlation between inconsistency, incorrectness and incompleteness problems in order to make the maximum use of the information available,
• the utility of the method in supporting group decisions under uncertainty by recording information about the participants, and
• the use of a natural language environment such as LOLITA.

5. The Limitations of the Approach

The Viewpoint Control Method is domain-independent but it is more useful for situations where most of the information comes from people and where there are few constraints imposed by the

environment in which the software will operate (a typical example is the requirements engineering needed to build decision support software to be used by a group of managers). Brackett [16]

found that the fraction of requirements elicited from people increases as constraints on the software requirements process decrease.

Although the method is not dependent on the quality of the viewpoints, as is the case for [6,7], nor on the language used to express the information, its analyses are restricted to the external features of the information: who said it , how it was said, and how does it relate to the existing information. Finally, support tools have yet as not been implemented.

6. Tool Support

Figure 2, shown earlier, can be seen as a high-level design for a support tool. Apart from supporting the use of the method, the tools will be useful for experimenting with the method in order to improve it.