1. Introduction

   In a world where markets are created and fortunes made in a matter of months, speed is of the essence. Systems involving software have traditionally been developed in a phased life cycle with coding following the preparation of design and requirements documents. Tests have sometimes not even been thought about until coding was well under way. This was perhaps fine when development lasted a year or more. But as product cycles shorten, the mismatch between what users want and what software houses can deliver has become increasingly uncomfortable.

   The situation is acute enough in consumer electronics, where growing complexity has to be traded off against demand for novelty. For instance, new models of mobile telephone appear every six months; already a typical phone contains a million lines of software.

   Electronic commerce resonates at an even higher pitch. Firms wanting to do business on the World Wide Web expect to get a working site up and trading within a month. Total time to market is less than traditional projects used to spend on starting up, fact-finding, and building teams.

   If high-tech projects are to succeed in such compressed timescales, they must have discipline and methods that work reliably. Above all, they must know what their users want, so they can implement it accurately.

2. Extreme Programming

   Extreme Programming (XP) aims to meet the challenge with a dramatically revised development cycle. The approach has inspired enthusiastic support, and equally vigorous criticism. Even the name, reflecting a hoped-for analogy with 'extreme' sports like snowboarding, seems calculated to provoke strong reactions.

   

   A recent book, Extreme Programming Explained: Embrace Change, by the inventor of XP, Kent Beck, provoked a stormy response. For example, review comments on the Amazon.com website range from 'unique, exciting, and inspiring' through 'a joy to read…Wow!' and 'seems like it would work with small teams of well motivated people' to 'an attempt to legitimize hacking and chaos'. Anything that attracts zealous converts, serious if sceptical practitioners, and hate mail – all at once – must be a dramatic set of ideas. What exactly is XP?

   XP has four main strands: listening, testing, coding, and designing, in no special order for they all happen in parallel. 'XP takes commonsense principles and practices to extreme levels', according to Beck. It uses its own variety of English: XP phrases with special meanings are italicized in this article.

   Continuous testing is a key feature: Beck insists that XP is a disciplined approach, and automated testing is essential – 'If you're not testing, it isn't extreme'. Two kinds of testing are distinguished. Functional Testing consists of a set of black-box, end-to-end tests owned by the 'customers' – apparently these include users, the terms not being too closely defined. A functional test is an executable script based on a use case [a scenario] supplied by a customer/user. Unit Testing, which as you might expect consists of detailed tests on the code, is owned by the developers.

   Functional tests have short names such as 'Holiday Pay' that mean something to the customers. The developers ask the customers what results they expect from each functional test. As soon as each test is defined it is added to the test kit for the project, and run at the end of each day's work. A characteristically idiosyncratic measure of XP project progress is the number of tests in the kit.

   Designing and Coding are similarly unconventional. Perhaps the nearest analogy is with the 'surgical teams' of expert programmers described by Fred Brooks in 'The Mythical Man Month'.

   

   Brooks was bothered by the 'tar-pit' of the software crisis of the 1960s and '70s, before modern specification and design methods had been devised. He noted that

   'one wants the system to be built by as few minds as possible'

   and suggested a team headed by two expert programmers:

   'the surgeon and the copilot are each cognizant of all of the design and all of the code'.

   XP similarly calls for pairs of multi-skilled programmer-designer-analyst-testers to work together – at the same screen – throughout the project. Obviously this promotes dialog: 'Requirements is a dialog, not a document' is a typically Delphic utterance from Kent Beck. Here he departs from Brooks, who wanted to support the pair of programmers with a team of writers and toolmakers.

   The dialog between developers extends to the customer/users as well. If there is no time – and perhaps no enthusiasm – for carefully prepared written documentation, then close co-operation is plainly essential. Since code is debugged and tested daily, users can be brought in often and shown working prototypes. Their comments, captured on cards, drive modifications and further development.

3. User Stories

   The one thing that can't be dropped amidst the noise of sacred icons being shattered is the need to know the goal: even XP developers have to talk to users to define the users' needs clearly. One of the key concepts in XP is the idea of the User Story. This is a simple scenario which can be quite concrete, if fictional:

   'Pamela is a 35 year old interior decorator who needs software to help her visualize color combinations'.

   Alternatively the scenario can be abstract:

   'When a communications link is lost the user is not notified, instead the system falls over and logs an error message'.

   XP developers take User Stories and assign them to Engineering Tasks, essentially work packages expected to take no more than a week. Larger User Stories are split up into several Engineering Tasks; smaller ones are lumped together. There are no discrete analysis, design, coding, and testing phases: instead, the pairs of developers intuitively, intellectually and pragmatically make the transition directly from User Stories to working code. Since each Engineering Task is sized to fit easily inside a pair of heads, the production of software is rapid and efficient. Since there is always somebody knowledgeable on hand to critique the design and code, bugs get weeded out at once rather than allowed to hide in dark corners.

   The attractive aspect of XP is the promise that user needs will be met accurately and quickly. The XP literature is full of phrases like 'short iterations' and 'rapid feedback' which imply that users are involved frequently.

   Detractors argue that if you go at a problem rapidly with an unstructured set of User Stories, there is a danger of solving parts of the problem but making an inflexible solution which cannot adapt to change. Maxims like 'do the simplest thing that could possibly work' do not make it sound as if XP is as effective as a more conventional approach. Traditionally, requirements engineers think through the scenarios – positive and negative – and organize them into a complete structure before coding. XP advocates reply that one more programmers' technique, refactoring, makes such analysis-in-advance unnecessary. Flexibility is restored by looking over the code and redesigning it so that it achieves its aims as simply as possible. That way, the code is easier to change and to use for other purposes. Badly written code is hard to maintain because similar things happen in several places, and all may need to be debugged and updated. Such 'refactored' code does each thing 'once and once only'. The approach is, like many things XP, interesting but controversial.

4. From Stories to Tests

   Customer/users are as always involved at both ends of development: saying what they want at the start, and testing to see that they have it at the end. In XP, Functional Test scripts are derived from User Stories; Alistair Cockburn, a use case guru, says that a user story is

   'a use case at 2 bits of precision. Bit 1 of precision names the goal of the use case, Bit 2 adds the main scenario.'

   Cockburn goes on to explain what he means by this: what the stories leave out is the whole question of failure handling – all the exceptions to the norm that make systems fall over. The criteria for acceptance may also not be obvious in a user story: XP handles this by having the developers discuss suitable functional (i.e. acceptance) tests with the users.

5. The Development style of the Third Millennium?

   Is XP the way of the future? The shape of software development has changed dramatically in the last decade, from systems built largely of C or older languages such as Fortran and Pascal, to systems built largely around off-the-shelf components. We can surely expect to see increasing use of commercial off-the-shelf tools, packages, and solutions. Already developers can produce e-commerce systems by clipping components together. These components range from ready-to-run servers and databases, through libraries of convenient Java objects, to complete business solutions. Other key components of client-server systems, such as a reliable, world-wide network with a universally-agreed set of protocols and naming conventions, of course already exist in the Internet and its many services such as the World Wide Web, DNS and FTP.

   Programming and design as stand-alone activities are on the decline. Happily, emphasis on the needs of users, customers, and other stakeholders is very much on the increase. Is XP the inevitable development style in this brave new world? Perhaps it is, at least for small to medium systems.

   XP does not claim to be suitable for the very large systems discussed by Fred Brooks. Brooks himself suggested that his surgical teams should be used as components of a tree-like project structure to allow large projects to work effectively. In those complex and delicate cases, developers will continue to need detailed user requirements, specifications, design documents, and the rest. XP aficionados may need to think through how they would scale up their approach for large and perhaps especially for mission-critical systems.

   Equally, it is not clear how XP might be applied to product development, where users and customers number in the thousands – or in their millions. XP seems to be targeted on custom-made software for a specific company. The developers and users can sit round a table or at a screen, and talk. When the voice of the users has to be transmitted through a helpdesk and support team, a marketing department, and a product management team, the immediacy of XP may be difficult to sustain.

   XP is explicitly about software development. Systems that require custom hardware inevitably demand specialization – hardware and software engineers, ergonomists, and so on. Teams of specialists could perhaps fit more easily into Brooks' model than into the Extreme Programming approach.

   Elsewhere, User Stories and Functional Tests may be all that are needed to document software projects on their meteor paths. Stories and test cases will describe the users' requirements: the code will do the rest.

   © Ian Alexander 1999 and 2004

   Readers who would like to find out more about XP should look first at the Wiki Wiki Website hosted on http://www.c2.com . It provides a rich hypertext based on debates about XP which readers can comment on or add to as they see fit.

   My book Scenarios, Stories, Use Cases contains a chapter by Kent Beck and David West in which they explain how User Stories work in practice; the book also discusses the trade-off between agile methods like XP and more conventional approaches such as use cases and Rational's RUP.

Ian Alexander's Home Page