Systems development has moved from treating systems as purely technical problems, towards seeing systems as parts of the users' environment. Stakeholders such as operators are increasingly involved in requirements engineering in particular.

Requirements engineering is balanced between co-operating with the stakeholders who own the requirements, and the need for a controlled process of development. The dilemma is that control and predefined methods tend to exclude co-operation.

This paper assesses progress towards the possibly unreachable goal of fully co-operative requirements engineering. Some leading methods are compared. The potential for Co-operative Inquiry to resolve the dilemma of co-operating manageably is discussed.

Users are becoming more involved in requirements engineering. Methods of obtaining requirements are becoming more co-operative [e.g. Macaulay 1993, Potts 1994, Robertson 1997]. These methods agree that requirements engineers should aim to facilitate rather than control. Most writers and practitioners also agree that users must own their requirements [e.g. Stevens 1998, p293].

But developments in methods and techniques [see e.g. Kotonya 1997] risk concentrating much of the power to decide what information to collect, and how to organize and validate it, in the hands of engineers. There is a danger that users may be confronted with carefully worked-out methods which predefine every process. Sophisticated types of diagram and notation also tend to exclude users. The dilemma is as follows: be too prescriptive, and co-operation is inhibited; be too free, and chaos results. This paper attempts a resolution of this dilemma.

There is good evidence [Standish Group 1998] that the primary cause of development project failure is lack of clear agreed requirements. This suggests that the crisis in the development of complex systems can only be solved when stakeholders themselves specify their requirements. They alone are the 'experts' in their own fields. They have the direct interest in the results of a successful development, which can give the determination to see projects through.

Users may lack the skills of the requirements engineer, so we have a specialized role: to assist and facilitate. In case this seems self-evident, note that the words 'facilitation' and 'co-operation' do not occur in the indexes of two recently published textbooks entitled 'Requirements Engineering' [Wieringa 1996, Kotonya 1997]. The paradigm of co-operation is far from fully established. This may be because co-operative methods are seen as too risky in a business environment. A method that allows stakeholders to speak out, but which is robust enough to survive challenges, is required.

Traditional systems analysis [see Maguire 1997, Wieringa 1995 for evaluations of RE methods] treated users as sources of information. These sources could be mined by surveys with questionnaires or structured interviews, or techniques of measurement and experiment such as on user interface or network traffic. Users had little opportunity to validate systems until acceptance testing, by which time errors of specification were costly to correct. There have been numerous spectacular failures of large public projects; these probably represent only the tip of the iceberg, since most privately-funded project failures remain concealed [Standish Group 1998].

System users and their actual problems can be studied directly by ethnographic observation. Complex domains like air traffic control [Hughes 1992, 1995] can repay careful study, as subtle details may be critical to a system's usability. Another method, Participant Observation [Flick 1998, p141ff] may also help engineers understand a problem and communicate with stakeholders.

The strength of observation is that tacit knowledge [Göranzon 1988, pages 53ff, 130ff; Polanyi 1962], such as the domain expertise of an air traffic controller, can at least be approached. Kotonya lists several studies which show that ethnography can capture requirements not otherwise elicited [Kotonya 1997, pages 67-72].

The weakness of ethnography and observation techniques, if used by themselves as requirements methods, is that users do not directly share in writing or validating requirements. In addition, observation seems to be perceived as too expensive for all but the largest projects. This may be unfortunate: Josefson argues persuasively [in Göranzon 1988, pages 19-30] that the piecemeal automation of a profession such as nursing is ignoring its real requirements.

RAD is a popular approach combining requirements engineering with systems (usually software) development. The more frequent, and therefore improved, user feedback that it offers to developers is clearly valuable. Among the many current approaches are DSDM [DSDM 1998], Volere [Robertson 1997], and SOMA [Graham 1998]. These lay down processes and templates for each phase and document. Volere is being used in consultancy; DSDM is popular in the firms where it is used, such as British Telecom (BT), though Graham has criticized the method for its weak theoretical structure. Graham's SOMA approach is explicitly object-oriented, well supported by the SOMATiK tool. Certainly it is one of the few approaches which can be followed right through the life-cycle. It is in use for financial trading applications.

Co-operation through interaction is a definite goal of all RAD methods. In contrast, the methods tend to be explicitly defined in advance, so users may have little influence on the processes they have to follow.

Checkland’s Soft Systems Methodology (SSM) [e.g. Checkland 1990] explicitly involves users in system development. His representations deliberately avoid analytic methods, favouring human views of the problem. SSM is not a development approach, though it has been incorporated into the Multiview method [Avison 1990]. Multiview pays commendable attention to obtaining viewpoints from different stakeholders before building system models. SSM also emphasizes ‘participative design’ based on Enid Mumford’s pioneering ETHICS method [Mumford 1985, 1996], not only involving users but reflecting their needs. Mumford’s work is consistently humane, far from the conventional focus on technology. SSM has influenced thinking on many other methods, e.g. VORD [Kotonya 1998].

Linda Macaulay’s User Skills Task Match (USTM) method [Macaulay 1993] is an effective technique for managing co-operative requirements capture. The need to address social issues is recognized, with the appointment of a facilitator expert in group management.

USTM provides a detailed structure of meetings, each planned and debriefed, and with a prescribed set of tasks and outputs. Users share in these tasks but are not involved in designing the task structure itself, though the agenda can be updated co-operatively. If any method is applied rigidly, the predefined structure itself may put up barriers to effective co-operation.

In USTM, the 'facilitator' is responsible for detecting, diagnosing, and solving any problem in the group. The facilitator thus retains power to control the group. Conversely the facilitator "must be external to the group and not a stakeholder" and does not share domain knowledge with other group members. Macaulay has applied USTM in several business contexts including the UK electricity distribution industry.

Colin Potts’ Inquiry-Based Requirements Analysis (IBRA) [Potts 1994] involves an inquiry cycle which seeks to support effective, conflict-free discussion. The aim is to build a requirements document which accurately reflects the "customer’s" needs. A process model is proposed but is not rigidly enforced, leaving welcome space for the users to negotiate the details of the requirements process.

The IBRA cycle can be matched with the CI cycle as a particular application of co-operative inquiry, namely the creation of the requirements document. The Potts cycle takes a document, allows the inquiry to challenge it (CI phase 3, Reaction), conducts a reasoned discussion (CI phase 4, Reflection) involving questions, answers, and reasons, which lead to a decision (CI phase 1, Proposition) on how to evolve the requirements. Scenario analysis is often applied to help users and researchers gain insight into the requirements. This leads to a change (CI phase 2, Action) to the requirements document, where the Potts cycle repeats. It is interesting that Potts has in effect ‘rediscovered’ CI in a specialized form. Potts uses artificial examples such as the meeting scheduler as tutorial material, but the workshop technique is popular and in use for consultancy.

IBRA users can share in designing the requirements process. As the engineer is probably not expert in the user domain, IBRA can be classified as a partial variant of CI.

It is an open question whether full CI [Heron 1996], with virtual equality of both knowledge and power between users and requirements engineers is either possible or desirable. Macaulay voices her doubts on the feasibility of fully co-operative meetings: "it is by no means clear that interaction between people with such a diversity of interests [as stakeholders have] would result in anything but chaos" [Macaulay 1993].

Initial personal experience of CI allays fears of chaos and wasted time: the method can deal effectively with issues on which group members hold strong and conflicting opinions. Meetings are orderly and can reach firm (and good) decisions rapidly. Without a method such as CI which directs attention to people as well as to technical issues, Macaulay's concern about full participation would be quite justified.

Potts' IBRA [Potts 1994] has several of the properties of CI, including a similar phase structure and the freedom for group members to diverge from the method. It also "directly supports inquiry and scrutiny", though "discussion can take place gradually and informally" instead of necessarily "in discrete bursts" in formal meetings. The danger of this is that individual stakeholders could feel excluded or outmanoeuvred if decisions seemed to be being made without them. Such feelings could lead to difficulties throughout a project, so Macaulay's doubts could be applied to Potts' method. In contrast, CI meetings can become quite informal, enabling "an inherently informal and situated activity" to take place naturally, but in a robust framework (Table 1) which can immediately address and resolve feelings of discomfort.

Full Co-operative Inquiry in our field could theoretically arise if

1. some users become skilled in requirements engineering, including the techniques of CI. This is likeliest in domains which already employ systems engineers.
2. Requirements engineers learn CI, and have the opportunity to work for an extended period in one user domain. This is likeliest in system houses and consultancies oriented to a vertical market.

Even if such perfect co-operation never materializes, the goal of helping users to get the systems they need seems clear. Especially where resources are limited, developers and users can get better systems by negotiating and prioritizing. For example, relaxing a performance constraint may save enough money to implement several much-needed functions. The developers may know what different approaches may cost; the users need to use this knowledge to decide which would be best for them.

The cost of all user-centred methods is similar, involving the time of workshop attendees, the effort to set up meetings and to share results. [Potts 1994] shows that these costs compare favourably with the costs of traditional analysis, and seem to give much better results.

Robust co-operative methods should improve communication between stakeholders (including developers), the requirements, and the resulting systems.