The Methodology Space

Citation preview

Downloaded from the Internet Archive: https://web.archive.org /web/20000819045916/http://members.aol.com/acockburn/papers /methyspace/methyspace.htm The Methodology Space Alistair Cockburn [email protected] Human and Technology Copyright notice: This is Humans and Technology technical report HaT TR.97.03 (dated 97.10.03). It may be submitted for reviewed, dead tree publication, at which time the copyright may transfer to that publisher. This copy is being web-published for early peer dissemination. Note from the author (Alistair): Between October 1997 and October 1998, I have extended the model somewhat, to include cultural aspects of teams. This then, represents the first pass at a description of Methodology. counter is since resetting March 15, 1999 Abstract Many people are unclear as to what a methodology is, what it should contain, what it should optimize, or how it should be used. Discussions about methodology tend therefore to be emotional and nonproductive. The purpose of this paper is to examine the space of methodologies, to provide a basis for emotionally neutral discussions about where any nominated methodology fits. It examines what one is and looks like, why multiple ones must and will coexist, what principles apply, what one might optimize and where one might be applicable or not applicable. No particular methodology is espoused in this paper, since it aims to provide a neutral basis for discussion. Section 0. Context, Problem, Main Result, Sequencing Context into which this paper fits At the current moment, early 1998, methodology discussions proceed without much in way of ground rules, hence also with emotional charge and often without progress. This paper addresses the ground rules. In the OO community, the word "methodology" is often used to mean a set of (mostly drawing) standards of the type UML, Shlaer-Mellor, OMT, Booch, or Coad. It may also indicate a set of design techniques such as CRC cards, role modeling, or semantic modeling, or transformational design. People who use the term in this way argue over how best to draw, how best to design, and whether such minor parts of project life are even worth arguing over. In large consulting houses, the word "methodology" means the role 1 von 27

definitions, job descriptions, activities and exchange deliverables of everyone on the project. Some consulting houses have rows of ring binders describing their methodology. Some can generate customized rows of ring binders on demand from a tailorable repository. Both uses term cast emotion (usually fear and loathing mixed with hope) into a discussion. People on projects recognize that they need some element of a methodology, but typically feel that the first definition of methodology represents only a small part of their work, and the second will be stifling. The result is distaste for the discussion of methodology, and typically unproductive dialog when such a discussion starts. The Problem this Paper Tackles This paper addresses the more serious matter of creating a fair dialog between people of opposing views, and the interesting but less critical matter of creating a taxonomy for methodologies. The paper succeeds if two people can carry through a conversation about methodology, understanding that they will disagree on final choice, while understanding each other's vocabulary, context and value system. It addresses the following specific questions: What is a "methodology?" What are its bounds, what are its dimensions? How might one fruitfully compare two methodologies? Is there one or must there be many? If many, how would one know which one or elements of one to adopt? Is one "better" at some thing or in some way than another? Must they be big, and if so, how should one best deal with their size? On the way, the paper splits open the methodology space and claims that certain questions dominate. For each question, it contains a proposed answer. I do not claim that the answers are correct, because there is so little hard data on these questions. I do claim that these are the dominant and interesting questions, and do assert that these answers are consistent with several decades of field work in software development. My hope is that by presenting the questions, and my best current answers, other researchers will investigate with proper social research methods, and substantiate or repudiate the answers I propose. Within their work I hope that the accuracy of my selection of questions will be examined. Main Result and Contribution The main result of the paper is that there are necessarily multiple methodologies, using the large consulting-house version of the word. The applicability of the methodologies sort along two essential dimensions: staff size (the number of people on the project) and system criticality, the potential damage of undiscovered system flaws. Dividing size into seven buckets, and criticality into four, we see 28 differently characterized methodologies. The number could be expanded without difficulty.

2 von 27

For any point in the size/criticality space, the designers of the methodology (a) select a scope of concerns to address: which project roles, activities, deliverables, and standards to cover and which to omit, and (b) work from their beliefs, previous experiences, fears, wishes, and philosophical principles, attempting to optimize some quality of the project. Methodologies therefore differ by the size, criticality, scope, optimized quality, and the grounding beliefs of the authors. Comparison of methodologies should focus on these different dimensions, and their relationship to the needs of the project or organization. The paper contains several secondary results. The first is the identification of nine essential topics contained in a methodology. The second is a set of questions and curves that can be used to discuss the optimizations of the methodology. The third is what to do with the bulkiness of a methodology. I shall suggest that most of the bulk should go into training courses, and the rest should fit easily onto a few sheets of paper per person. The sheets name the reviews the person must attend, and the standards they must adhere to. Very few people should have to sit down and read the entire methodology. Structure of the Paper This paper is structured to follow the way I see people having an open dialog on a shared interest: People working through an open dialog topic first try to align their vocabularies. So in the first section I set down the vocabulary as I intend to use it, with some attention to how other people use it. This section covers the basic terms plus the questions, "What is a methodology?", and "What are its bounds, what are its dimensions?" That being done to the extent possible, people then often see that they differ on the scope they wish address. Once they allow for their slightly different scopes, they often see that they have no basic difference of opinion and can work in future in harmony, mapping scope and vocabulary as they talk. Second, therefore, I discuss possible scopes of discussion, and provide a way to talk about the scope of any given methodology . After vocabulary and scope have been aligned, the next differences tend to be irreconcilable. They are based on philosophy or principles the people have come to adopt, often as a result of their personal experience. Some small amount of conversion can be done on the basis of trading experiences, but there is typically no reconciliation or further convincing once the guiding philosophy is reached. Third, therefore, I discuss some principles of methodologies. The principles I discuss are those that match my personal experience, and I make this quite clear. There are, to date, no hard data on these topics, so the principles are put forth in general and strong terms. A plausible discussion of methodologies might call these principles into question 3 von 27

and put forth alternative ones. I do claim that the points of discussion that I name are key points of discussion on the topic of methodologies. Finally, I discuss anchor points in general. It is important to discover them, because is so hard to shift someone's anchor points. There are usually several anchor points: a quality the methodology author wishes to optimize (time-to-delivery, visibility, or freedom from defects, for example), the personal fears, wishes, and philosophy of the methodology author. After going through the four dialog layers, I discuss the apparent necessity for many methodologies, a plausible way to deal with that multitude, and what to do with their bulkiness. Section 1. Development of Vocabulary Terms methodology The American Miriam-Webster dictionary defines "Methodology" as: 1) a series of related methods or techniques; 2) the study of methods. The Oxford English dictionary has only the second definition, the study of methods. I use the American form in this paper. method "Systematic procedure..." Synonymous with "technique" in this paper. One way to think of the difference between a methodology and a technique is that while one or two people are working alone in a room, they are using a technique; if 20 people are working, they need a methodology. The difference lies mostly in the communication needs. I develop the idea of communication as essential to methodology throughout the paper. precision How much you care to say about a particular topic. Pi to one decimal place of precision is 3.1, to four decimal places is 3.1416. Source code is more precise than its class diagram, and the assembler code is more precise than the high-level source code. Methodologies manage the evolution of precision over time. Some call for more precision earlier than others, according to the author's fears and wishes. accuracy How correct you are when you speak about a topic. To say "Pi to one decimal place is 3.3" would be inaccurate. The final object model needs to be more accurate than the initial one, and the final GUI description is more accurate than the first, low-fidelity prototype. Methodologies manage evolution of accuracy as well as precision. relevance Whether to speak about the topic or not. GUI classes are not relevant to the domain model. Infrastructure design is not relevant to collecting user functional requirements. Methodologies typically strive

4 von 27

to separate work topics by relevance. tolerance How much variation is permitted. Tolerance applies to standards. In coding, a revision code may be required, or it may be left to vary by tool. In writing requirements, designs or programs, the naming, the line breaks, the indentation, may be specified or left to the peoples' discretion, or acceptable bounds may be stated. An example of tolerance in a standard for incremental development would be that a working release must be available every 3 months, plus or minus one month. scale How many items are rolled together to be presented as a single item. A "class category" is a scaled view of a set of classes. Scale interacts with precision, in that the dot density of a screen or printer limits the amount of detail that can be put onto one screen or page. However, even if it could all be put onto one page, some people would not want to see all that detail. They want to see a rolled-up or high-level version. Project plans, requirements, designs, all admit of a scaled view. As with geographic systems, maps and mechanical construction, it is possible to put a scale on the deliverable, although it is less consistent. A class category represents a scaling of 1:4 all the way up to 1:30, depending on the number of classes in the class category. methodology weight The combination of how many elements the methodology specifies, how precisely it specifies them, and with how little tolerance. Conceptually, the product of its size and its specific density (conceptually only, because I do not attach numbers to size and specific density). methodology size The number of control elements in the methodology. Each deliverable, standard, activity, quality measure, technique description, is an element of control. Some projects and authors will wish for small methodologies, some will wish for larger. methodology specific density The amount of precision and the tightness of tolerance in the methodology. Greater specific density corresponds to tighter controls. A team developing a life-critical system may select a methodology with tighter controls, i.e., higher specific density, than a team developing a commercial system. This also varies according to the author's fears and wishes. criticality The nature of damage of undetected defects. In this paper, I classify criticality simply as one of: loss of comfort, loss of discretionary money, loss of irreplaceable money, loss of life. Other classifications would be possible. staff size The number of people needing to be coordinated on the project.

5 von 27

problem size The number of elements in the problem and their inherent cross-complexity. My carefully considered view is that there is no such thing as an absolute measure of problem size. Often a new person will see a simplifying pattern that reduces the problem size. Difficulty in settling on the problem size transfers into a corresponding difficulty over the staff needed and the methodology weight needed. project size Synonymous with staff size (in this paper). People sometimes expect project size to correlate with problem size or problem complexity, but that is a flawed line of pursuit. There is considerable disagreement about what staff size is needed for any perceived problem size, and there is no agreed-upon metric for problem size. I review the matter of problem size, staff size and methodology size later in the paper. What does a Methodology consist of? Sam Adams coined the terms "big-M" methodology, when referring to the large consulting-house usage, and "little-m" methodology when referring to the popular OO methodology books, written by Booch, Coad & Yourdon, Rumbaugh, Wirfs-Brock, Shlaer & Mellor, and others (personal communication). I have found value in these terms and shall use them in this paper. "Little-m" methodologies describe some deliverables with their standards, and perhaps a few techniques, for primarily one role on the project, the designer. In terms of the definitions given earlier, that usage can well be accounted by the word "method" or "technique", plus notational standards. "Little-m" methodologies are interesting to consider, since they receive so much attention in most discussions of methodologies. In the context of the results of this paper, "little-m" methodologies have a much smaller scope than "big-M" methodologies. For this reason, I shall primarily work with "bigM" methodologies in this paper. Methodology is how an organization repeatedly produces and delivers systems. It is who they hire, what they hire them for, what people expect from co-workers, what conventions they follow, and even what sorts of projects they agree to do. When an organization places a job advertisement in the newspaper, the ad is an artifact of that organization's methodology. The ad names the job title, duties, and skills expected of the person to be hired. It is already understood that the hired person will use certain skills to carry out certain activities, producing certain deliverables, interfacing to certain other people. This discussion informs us as to what a methodology consists of (see Figure 1):

6 von 27

Figure 1. Nine of the 10 methodology topics with samples (Values not shown)

1. Roles - the job descriptions put into the ads when you need to hire more staff: JAD facilitator, project manager, requirements gatherer, business class designer, tester, program designer, programmer, writer. The methodology may divide a job description into several pieces, i.e., roles, if the methodology permits clustering the same roles differently in the same project. 2. Skills - the skills those people should have when they answer the ads: facilitating, planning, programming, OO design, familiarity with tools. 3. Techniques - the techniques the person uses in their work: JAD (Joint Application Design), session facilitation, CRC (Class-ResponsibilityCollaborator) card exercising, Smalltalk or C++ programming, domain modeling, use case modeling. 4. Teams - how you group the people and how you assign people to roles. 5. Tools - what tools the people use in their jobs, either within a technique or to produce a deliverable according to the standard. 6. Deliverables - what you want each person or team to hand off to another person or team: use cases, class-, screen-, test- specifications, framework documentation, class diagrams, interface diagrams. Typically, a deliverable can be written in a variety of ways using a variety of notations. 7. Standards - what is permitted or not permitted in the work. There are notational standards, decision standards, and weaker standards, such as 7 von 27

project conventions (see discussion below). 8. Activities - what meetings, reviews, milestones and other general activities the person must attend, generate or do. There is naturally the activity "create deliverable", for each deliverable. The more interesting activities are those that involve communication, such as reviews and declarations that a milestone has been reached. 9. Quality - what rules, issues or concerns are to be tracked for each deliverable. In this paper, I associate quality concerns with the resulting deliverables. Some people prefer to attach quality discussions to "Activities". Whichever is chosen, the methodology addresses quality issues, and the differences in approach will show up in the different methodologies. 10. Values - what values drive the team. The team values affect all of the first nine characteristics, particularly, how closely they will follow standards, what activities they will accept, how closely they will work with each other, what underlying philosophies of software development they will accept or reject. Notational standards legislate several things, one of which is the selection of drawing style for class diagrams. This is where the Fusion, Ptech, Booch, OMT, UML discussion most often resides. The person uttering the phrase, "UML won the methodology wars," is referring to selection of notational standards. The authors of UML are aware of this, and were careful to define UML as the Unified Modeling Language. Other decisions in the methodology that count as notational standards are selection of a programming language (e.g., ANSI C, Java, etc.), selection of a particular format for use cases. Decisions about standards operate at different degrees of precision. Selecting the programming language C++ is a decision, but not yet precise enough. The next degree of precision would be to require, say, ANSIstandard C++. The next level of precision in the coding standards would specify standard header formats, commenting requirements, formatting, and so on. Not all of these need be stated; there is an interplay between precision and tolerance. That which is unstated remains open, i.e., a wide variation is tolerated. Similarly, selecting UML as a class diagramming standard is not sufficiently precise for most projects. There is the question of whether to subset it, whether to capture the author's name and the date on each sheet, the use of off-page references, and so on. Decision standards are those that affect the decisions that can be made. A possible decision standard is to require that each project increment run in less than four months, with two iterations per deliverable. A possible decision standard is to require that any new technology selected be supported by at least two vendors. 8 von 27

The methodology may leave certain standards open, to be determined as project conventions. These may include precise formatting of use cases, class diagrams, UI specs, and program code. The people on the project would then get together to establish some conventions they will follow, as a completion of the methodology. Summary. A methodology is an agreement of how multiple people will work together. It spells out what roles they play, what decisions they must reach, how and what they will communicate. A person working alone may still choose to follow a full methodology, and will then play the various roles in turn. Section 2: Methodology Scope It follows quite naturally from the discussion of the term, "methodology", that the scope of a methodology is the range of roles, their activities and their deliverables it attempts to cover.

Figure 2. The three dimensions of scope. A methodology selects a subset of all three. The first wave of OO methodologies, written up through 1991, targeted the OO designer as the key role, and discussed the techniques, deliverables and standards for one activity of that role. They fell short of the needed methodology descriptions in two ways. First, the scope was not as broad as was needed. A real project involves more roles, and each role involves more activities, more deliverables and more techniques than these books present. Then, they were typically too constricting as well. One single design technique, as those books presented, does not work for all the people

9 von 27

targeted. A second wave of published OO methodologies began with Objectory and Fusion, and now includes the OPEN methodology. These began to treat the life cycle from requirements through test. Meanwhile, groups with long continuous experience, such as the U.S. Department of Defense, Andersen Consulting, Ernst & Young, James Martin and Associates and IBM already had methodologies covering the full lifecycle, some even starting as early as project sales and project setup. These methodologies might cover every person needed on the project, from staff assistant through sales staff, designer, project manager, and tester. Scope, then, fits into three dimensions: lifecycle coverage, role coverage, and activity coverage (see Figure 2). Life cycle coverage indicates when in the life cycle the methodology picks up, and when it ends. Role coverage is which roles fall into the domain of discussion. Activity coverage is which activities of those roles fall into the domain of discussion. The methodology may wish to take into account filling out time sheets (a natural inclusion as part of the project manager's project monitoring and scheduling assignment), and may choose to omit requests for vacation (omitted because it is part of the basic business operations). One is unlikely to find a methodology that attempts to cover all activities of all roles. However, some existing methodologies do to cover all parts of the lifecycle from initial meeting or sales call through handover to a maintenance project, and all roles being paid out of project funds. A comparison of methodologies should take into account the intended scope of the methodology and the needs of the project or organization involved. "Little-m" methodologies revisited. Experienced developers, and the authors of the OPEN methodology, express a general sense of dissatisfaction with the little-m methodologies, saying that these do not cover enough ground to represent what is truly going on within the project, and do not represent the concerns that the developer typically encounters. In the context of the above discussion, we see that the "little-m" methodologies have a relatively small scope. Typically only one role is addressed, the domain designer or modeler. For that role, only the actual domain modeling activity is represented, and only during the analysis and design stages. Within that very narrow scope, typically one or a few techniques are described, and a few deliverables are outlined. For those deliverables, standards are given, but without review activities or quality measures. All of this does not say that little-m methodologies are bad in any way - only that their intended scope is narrow compared to the full activities

10 von 27

the developers are experiencing. The OO industry is already changing with regard to the scope of the methodologies being produced. People who have experienced several full projects are now producing methodologies with larger scopes [XP, RDD]. Where it is not the purpose of this paper to assert whether they are good or bad, it is the assertion of this paper that the above framework for discussing scope will assist someone in evaluating the methodologies. The OPEN methodology Ian Graham, Brian Henderson-Sellers, and Houman Younessi, in The OPEN Process Specification [Graham97], give their definition of methodology:

An (OO) method or methodology (we will mostly use the words interchangeably) must cover the whole lifecycle both technically and managerially. That means a methodology must provide, at least...: * a full lifecycle process * a comprehensive set of concepts and models, all internally selfconsistent * a full set of techniques (rules, guidelines and heuristics) * a fully delineated set of deliverables * a modeling language which includes a metamodel and a workable and intuitive notation * a set of metrics * quality assurance * coding (and other) standards * reuse advice * guidelines for project management (including team structure and roles, procedures and resource allocation and management). If possible, a supporting set of tools should be available. (from [Graham97], pp.2-3)

Their requirements for what may be considered a "methodology" are 11 von 27

considerably more demanding than those of this paper. They require that the "full lifecycle" be covered, they require a metamodel, they require a modeling language (and they require it to be 'intuitive'). Although "full lifecycle" is a term with different meanings in different organizations, their book names the tasks included within the scope of OPEN, and so their own intended scope becomes clear. Their requirement for a metamodel may represent an attempt to optimize something with the methodology, although they do not name what they intend to optimize. The requirement for a (intuitive) modeling language reflects some of their personal experiences, fears, wishes, and principles. As the OPEN methodology is being published in a series of books, it make a good candidate against which to compare other methodologies, using the dimensions provided in this paper.

Figure 3. Scaled overview of a methodology, showing only roles, deliverables, milestones.

What does a methodology look like? A methodology is mostly text, describing techniques, activities, meetings, and quality measures. Samples of these elements can be seen in the books, Object-oriented Methods, Pragmatic Considerations [Martin96], and The OPEN Process Specification [Graham97]. Many of the standards take shape as forms to be filled in or examples of deliverables. A sample of deliverables and standards can be seen in the book, Developing Object-Oriented Software [OOTC].

12 von 27

At some point, a scaled overview of the entire methodology might be produced, which may look like Figure 3. This view shows the interaction of key events in the life of the deliverables, associated with the roles who produce and review the deliverables. This view can act as a table of contents to the methodology. A separate description of each role, each deliverable and each event or milestone needs to accompany it. A fully defined methodology tends necessarily to be thick, as the following calculation illustrates. Imagine there are nine roles defined in the methodology. Each is responsible for four deliverables. Each deliverable has one standard and goes through three milestones, such as design reviews and publications. Then there are 9 x 4 x 3 = 63 elements of the methodology to be defined and correlated! This does not even touch techniques, quality issues or other activities. Two relevant questions appear at this point, which I must mention but do not intend to address further: How does one present a methodology? The current standard is to use the system description techniques of computer science, perhaps process-based or object-oriented. This is satisfactory to followers of what Pelle Ehn calls the Cartesian school [Ehn93]: software is an object result produced by a construction system (of people), ergo, objective, system description techniques are an appropriate way to describe development methodologies. Such descriptions are not satisfactory to people who view software development as a craft or apprenticeship business [Lave91], as an evolving "language game" [Ehn93], or as a "participation framework" (an informal and emergent structure of social interaction, in which all of the participants' continuous assimilation and ongoing evaluation are crucial to moving it forward) [Goodwin84]. To people of these schools, a system-oriented description of the methodology is not so helpful, since the methodology evolves and transmits by watching, doing, and collaborating. How people stand on these sorts of issues affects their willingness to participate in discussion of a methodology, and what questions they believe are pertinent. How does one debug a methodology? A methodology describes the workings of numerous (notoriously variable) people, over relatively long time periods. How can we tell that all the pieces connect, and that all the pieces that are needed are present? To the extent any part is being proposed for the first time, how can we tell whether that part will work? Does the methodology fairly predict what the people will encounter on the next project? Debugging a methodology falls outside the scope of mapping the methodology space, but is a serious and difficult issue. Section 3: Principles A methodology will use principles selected by its author. Those principles 13 von 27

may come from a hidden background, namely from the author's experiences. In this section, I name some of the issues I have come to consider key discriminants of methodology. I also make assertions that I think are true, related to those issues. People having an open dialog on methodology should expose their experiences, their preferred principles regarding methodologies, and discuss whether they believe my assertions or not. Researchers may investigate whether the assertions are true. (Truth in advertising: The principles in this section primarily come from my investigation of several dozen projects [Cockburn94, Cockburn98]. There is a shortage of reliable research results on these issues. Therefore, while I make the claim that my assertions on each issue are plausible and reasonable, I make the stronger claim that the issues are the correct ones to discuss, and that it is worth the effect for researchers to get statistically significant data on them. ) The principles and assertions I propose that apply to methodologies in general are these.

Figure 4. Efficiency of communication Principle 1. Interactive, face-to-face interactive communication is of a higher grade than any other for the purpose of evolving and communicating designs. The characteristic of being interactive is more important than that of being face-to-face. This is illustrated in Figure 4. The figure shows a dropping off of effectiveness in the communication as the participants more from standing in front of a whiteboard, for example, to talking on the phone, to an email chat session, to a videotape, to a written document. Note that effectiveness is always "with respect to a purpose". For other purposes, such as making a

14 von 27

clear contractual agreement, the curve would be different, even reversed (see below). The assertion in principle 1 implies that people sitting near each other and with frequent, easy contact, will find it easier to develop software, i.e., the software will be less expensive to develop. It implies that as the project size increases and interactive, face-to-face communications get hard to arrange, the cost of communication goes up while the quality goes down, so it will be harder to develop the software. It does not say the quality goes to zero, nor does it imply that all software can be developed by a few people sitting in a room. It implies that a methodology author might wish to emphasize small groups and lots of personal contact if productivity and cost are key issues (as for example, in Extreme Programming [XP]). The assertion is supported at the research level by [Plowman95]. See also [Sillince96] for an discussion of different aspects of communication within an organization. Alternatives formulations to principle 1 are possible, for example, that written communications are more effective for the purpose of evaluating contractual conformance. Use of this formulation would drive the methodology authors to emphasize written documents if they foresee legal liability as a key issue.

Figure 5. Effect of project size

15 von 27

Principle 2. A larger methodology is needed when more people are involved (see Figure 5). Larger means containing more control elements. The way to read figure 5 is as follows: Communication load rises as the number of people involved increases. Effectiveness per person drops as the communication load increase. Since methodology is a matter of coordinating the people and managing the communication, its size must also rise with the number of people involved. However, both necessarily and fortunately, it rises slower than the communications load. Fortunately, because otherwise it would not offer relief from the communications load, and necessarily, because it describes interactions between roles and deliverable types, not role instances and deliverable instances. It grows in size as the number of roles and deliverables types increase [Harrison96].

Principle 2 discusses the relationship between project size and methodology size. The assertion tells us that one should not expect a small-team methodology to work properly for a big team, and one need not use a bigteam methodology for a small team.

Figure 6. Time cost of growing methodology size. Principle 3. A relatively small increase in methodology size or specific density adds a relatively large amount to the cost of the project. (see Figure 6) This is one of the most significant, and may be the most contentious of the assertions in this section, and has not been properly researched, to the best 16 von 27

of my knowledge. Personal observation and project interviews indicate that people feel that stopping normal discussion and development work to attend to legislated coordination activity or coordination deliverables, costs a lot in terms of lost concentration, and in rework of the coordination deliverables as the projects shifts. Coordination deliverables include requirements documents, design documents, project plan, and test documentation. The principle does not address whether the coordination activities and deliverables are good or bad, considered beneficial or hazardous. It addresses the cost of adding elements and control to the methodology. It would be good for the industry if research was performed to detect the time cost of methodological elements, and compare that with the time cost of repeated informal discussions. At this point we can use principles 1 - 3 to examine the relationship between methodology size vs. project size vs. problem size. Most people find this discussion tricky, since there is a tendency to think that a larger problem must be solved by more people, i.e., as problem size increases, methodology size must also increase. I hope the following discussion, Figure 7 and Figure 8 show the flaw in that line of thinking. The crux in the argument is that project size and methodology size create a positive feedback loop. With fewer people, less methodology is needed; with less methodology, the people work more efficiently; working more efficiently, they can successfully address a larger problem. So although there is a ceiling to size of problem a small team can attack, it is higher with a heavy methodology. Once more people are put onto a project, they need a heavier methodology to coordinate their work. The heavier methodology lowers their individual productivity, so more people are needed on the project. Since methodology size grows slower than project size, eventually they get to a point where they can solve the problem and manage the coordination activities (assuming sensible management). This does not mean that a small team can necessarily solve a larger problem than a large team. It does mean there may be an area of overlap, where a small team with a light methodology can solve the same problem as a larger team with a heavier methodology. This is shown in Figure 7.

17 von 27

Figure 7. Problem size and methodology affect staff. Figure 7 asserts two things: a) For a given problem size, you need fewer people if you use a lighter methodology, and more people if you use a heavier methodology. b) There is a limit to the size of problem that can be solved with a given number of people. That limit is higher for a large team using a heavier methodology than for a small team using a lighter methodology, i.e., at some point you will need both a large team and a heavier methodology. The trick with Figure 7 is that there is no known way to assess the actual problem size at the start of a project, and no way to determine the actual number of people needed to solve it. Our industry is littered with projects that guessed incorrectly. What Figure 7 does not show is, For a given problem size, how fast does the required number of people grow as the methodology grows? Where in the other graphs I have shown general curve forms, exponential, logarithmic, etc., for this question there is so much data missing that I cannot even hazard a general curve.

18 von 27

Figure 8. Methodology weight and problem size. Figure 8 examines the same topic from a different angle. a) At the left side of the diagram, it asserts that a small team using a light methodology will be able to solve larger problem than a large team using an equally light methodology (the large team will suffer confusion in their communications). b) At the right side of the diagram, it asserts that a large team using a heavy methodology will be able to solve larger problems than a small team using an equally heavy methodology (the small team is slowed down in methodology artifacts). c) The size of problem the small team can solve drops as the methodology gets heavier. The size of problem the large team can solve grows as the methodology gets heavier. At some point they cross. The exact shape of the curves and the crossover point are not asserted, not only because I don't know them, but at least because they depend on the particular people, problems and methodologies involved. From the above discussion we can expect the optimum curve to be discontinuous. If there is a given methodology which is optimal for a given type of problem, its value will eventually drop as the team grows, and a different team size X methodology will become optimal. The industry is full of stories of the effectiveness of two or three people working alone. More recently, and dealing with a larger team size, the Chrysler Comprehensive Compensation experience [XP] was that eight people using an extremely light methodology successfully delivered what was considered a large system,

19 von 27

which a larger team with more normally sized methodology failed to deliver. The hazard here for people discussing methodology is that they usually have fixed assumptions about where those steps are located. The topic for researchers is to find anything more quantitative to say about this subject. Principle 4. Greater specific density (more publicly visible correctness) is called for in the methodology for a project with greater criticality (more damage can result from an undetected defect). This principle addresses the question of how much tolerance the methodology contains. If development costs increase rapidly as methodology weight grows, then there should be a strong disincentive to use a heavier methodology. However, there are times when the extra cost is worthwhile, life-critical systems being the obvious example. I have chosen in this paper to separate system criticality, the potential for damage, into four zones, although more could easily be chosen: loss of comfort, loss of discretionary moneys, loss of irreplaceable moneys, loss of life. A system operates in the loss of comfort zone if a failure means that people will have to go and do more work by hand, or call each other and repair a miscommunication. Examples might include programs that summarize reports, help with purchases, track sporting events. A system operates in the loss of discretionary moneys zone if the loss of money or related valuables is merely uncomfortable. Even paying people the wrong amount or putting the wrong amount on their invoices lies in this zone, since these can be repaired with extra work. A system operates in the loss of irreplaceable moneys zone if the loss of moneys or related valuables has effect corresponding to going bankrupt. A run on the national banks would fall into this category. A system operates in the loss of life zone if people are likely to die from a system malfunction. An atomic power plants, the space shuttle, airplane control systems, fit into this category. The assertion is that the software development process can justify greater development cost to protect against mistakes, as the system moves from zone to zone. The damage from a latent failure in software to control an atomic power plant is much greater than software to track my weekend bowling matches. Accordingly, the methodology used to build the power plant software can afford to be much more laborious and expensive, if the added expense reduces defects. The principle relates the specific density of the methodology to the software system's criticality. Specific density is the amount of precision and the tightness of tolerance in the methodology; greater specific density 20 von 27

corresponds to tighter controls. To get a feeling for increased specific density, consider two methodologies. Both call for use cases. The methodology for the bowling software accepts that the use case might be simply a few sentences, a little story, written in any convenient place - on the board, a scrap of paper, a word processor. This methodology for the power plant, however, insists that the use cases be written in a particular tool, with very particular fields being filled in, version controlled, etc. It would further call for review, change control, and sign-off of the use cases at several stages in the life cycle. The grammar to be used in the use case text is also legislated. The cost to develop the second use cases is greater per use case, and the benefit expected is that more writers and readers will be able to collaborate, and fewer mistakes will be made. The extra cost is justified by the added safety of the final result, and the communications cost within the larger team. Both methodologies call for use cases, but the second exerts tighter controls on just what passes the test of being an acceptable use case. In it, use cases are described to greater precision, and the tolerance for variation is smaller. In the terms of this paper, the second methodology has greater specific density. The total weight of the methodology is its number of controls (size) times their specific density. There is a hazard of accidentally mixing project size and system criticality when comparing the value of methodologies. Here is an example in which size is held constant, contrasting only criticality. It should be immediately clear that the more critical project can cost-justify methodology elements with greater specific density: a) Two people jointly develop the software that controls the movement of boron rods in an atomic power plant; b) Two people jointly develop the software for ordering late-night food from the company cafeteria. Section Summary. The four principles introduced in this section separate methodologies by communication style, development cost, team size and system criticality. The principles show there to be a relationship between the topics. The particular assertions made about those topics are that development cost goes up as team size goes up, as methodology tolerance goes down, as communication moves from face-to-face to written form; that the increased costs are justifiable as criticality goes up; they are necessary as team size goes up; that certain problems need the larger team sizes anyway. People comparing methodologies should attempt to make explicit their

21 von 27

assumptions on these issues. Even if they disagree, they will at least understand each other better. Section 4: Anchor Points Each person has had experiences which inform their present views, which serve as their anchor points for the discussion. As a person's anchor point is not likely to be moved by the discussion, an open dialog on methodology should expose the anchor points and key experiences. Anchor points come in at least these categories: personal anchors (experiences, fears, wishes, philosophy), and optimizations. Personal anchors When two people discuss a methodology, they often justify their point of view by relating to a particular project or experience. These experiences sometimes reinforce an already existing view they hold, but sometimes form their present view. Telling the stories permits one person to see a bit of how the other person came to their views, even if both people interpret the story in different ways. The stories help draw out the author's fears, wishes, and reasons for selecting a particular optimization. "All methodology is based on fears," wrote Kent Beck in a discussion of methodology. Although this sentence appeared merely dismissive at first, I have found it to be largely true. Examining the methodology from this perspective, one can almost guess at the past experiences the methodology author has had. Each element in the methodology is a stop against a bad experience some project has had. Afraid that programmers make silly little mistakes? - Hold code reviews. Afraid that users don't know what they really want? - Create prototypes. Afraid that designers will leave in the middle of the project? - Have them write extensive design documentation as they go. As a reaction against these fears, Kent Beck and Ron Jeffries instigated a "no fear" methodology, which they call Extreme Programming [XP]. It is based on the assumptions of small teams, close communication, and rapid development. The practices are intended to increase team comfort and ignore standard fears. The practices include programming in pairs, no individual code ownership, continual integration, regression testing at all times and no extra documentation. Where the OPEN methodology is a useful reference for comparing methodology structure, XP is a useful reference for comparing fears, because it builds on so few. Listeners almost always counter XP by stating their own fears, wishes and project experiences. Wishes are the counterpart to fears, they are the certainties we seek. A wish of XP is perhaps to reduce the "unpleasant" duty of documentation and double maintenance, to increase personal productivity and pleasure on the job. A wish of the OPEN methodology is to increase repeatability and visibility. Once the methodology authors state their wishes, much of the

22 von 27

design of the methodology becomes apparent. The final part in the personal baggage of the methodology authors is their philosophy. Some adopt a laissez-faire philosophy, some a military control philosophy. The philosophy comes with the person, shaping and being shaped by their experiences, fears and wishes. It is not likely to change quickly. It acts as a force on the shape of the methodology.

Optimizations All else being taken into consideration, there is still one dimension left for the methodology authors: what they wish to optimize in the development process. A methodology that chooses to optimize for cost will look different than ones optimized for visibility or defect freedom. For any pair of methodologies, the observer will have to take into consideration what they are each trying to optimize. In some methodologies it is easy to see what is being optimized. Both Extreme Programming and the methodology of [Cockburn98] claim to optimize for productivity and cost (and they look quite different, reflecting their respective fears, wishes, and target project sizes). The Personal Software Process of Humphreys [Humphreys97] optimizes for predictability. In others, it is harder to see. The OPEN methodology may be optimizing for program correctness, progress visibility, and repeatability, although these are not so clearly stated. The OO methodology in Martin and Odell [Martin96] is so general and tailorable that it is not clear what is being optimized, or whether optimization is left up to the individual project. Section 5: The Resulting Multitude of Methodologies The conclusion from the preceding sections is that there are many, legitimately different methodologies (see Figure 9). It is not the case that the national bank's Year 2000 project, the space shuttle software project, an overtime food request project, a promotion tracking project and a word processor software project should all run the same way. Their needs differ in three essential dimensions: The number of people involved. The criticality of the project. What the methodology optimizes for. The personal anchor points of the methodology authors then give further differences in the final form the methodology takes. In figure 9, I divided the space into seven project sizes and four zones of criticality. These are arbitrary but plausibly sensible divisions. The number of cells can easily be increased. I divided each dimension at places where my 23 von 27

experience indicates that the methodology is likely to take on a different nature, such as increasing the project size by 250%. Within each cell, a different methodology may be called for. Each cell admits of several methodologies, depending on whether the project leaders are in search of productivity, visibility, repeatability, or correctness. The methodologies will get larger (more communication elements) toward the right, and denser (tighter controls) going up, hence heavier moving up and to the right, or lighter, moving down and to the left. According to principle 3 above, moving to the right or up adds a large cost to the development of the project, so there is economic incentive to consider a project to sit as far to the left and down as possible (I recognize that there are different incentives, such as image and prestige, to consider a given project to sit further to the right and up).

Figure 9. Many methodologies, people X criticality X optimization. Section 6: Use of Methodology We have established that there are many legitimate methodologies and any one of them is going to be large. At the same time, there is a common experience that methodologies simply get ignored. They are widely perceived as large, fat, stifling, hard to learn[Cockburn94]. What is there to do about this? The answer comes in two pieces: separation by roles, and training. A methodology is a description of a subset of the jobs and activities of the people on the project. There are not many people on the project who actually need to read the entire methodology. The entire job of a single individual fits 24 von 27

within a small part of it. When a company puts an ad in the paper for a new employee, the description they put in with that ad is an artifact of the methodology. "Java programmer", "OO analyst / UML". These phrases already declare the partitioning of work, the job responsibilities of the employee, the skills deliverables and standards they will use. If the person shows up already skilled, there is little of the methodology they need to see. They will need to know the Java coding standards being used, who they get requirements from, how they deliver their code, etc. Then they simply get to work. They will not see the overall methodology for quite some time, and may never read it. In point of fact, there may be nothing to read, since the methodology may be contained within the cultural memory of the organization (see last paragraphs of section 2). For any one particular role on a project, the methodology contains a relatively bulky part and a relatively thin part. The relatively bulky part describes the techniques the person should be familiar with or is likely to use. The relatively thin part is the summary of standards to follow. The employee who knows the techniques will simply get to work, referring only to the thinnest part of the methodology description - the standards for that role. That leaves the bulkiest part of the methodology, the description of techniques. Although techniques are the daily activity of the people on the project, few people ever learn a technique on a project just by reading the description of that technique. They tend to learn either by working with someone who knows the technique or by taking a class in it. In a course there is time to devote to lecture, exercise and discussion. The techniques section of a methodology, therefore, serves to define a teaching curriculum of what to teach people, more than to be read and assimilated on its own. This section addresses the concern that methodologies are bulky and stifling. The bulkiness can be moved to course contents and spread across roles. The remainder is standards, not saying how to think, but how to communicate to colleagues once the thinking has been done. Summary A methodology contains nine basic elements: roles, skills, activities, techniques, tools, teams, deliverables, standards, and quality measures. The main result of the paper is that there are necessarily multiple methodologies, any individual one necessarily tending toward being large and bulky. Different methodologies are needed depending on the project size (number of people being coordinated) and the criticality of the systems being created. For any point in the size/criticality space, the designers of the methodology select a scope of concerns to address (which project roles, activities, deliverables, and standards to cover) and optimize some quality of the

25 von 27

project, working from their personal anchor points, including the experiences, wishes, fears and philosophy of the author. Methodologies therefore differ by the size, criticality, scope, optimized quality, and the personal anchors of the authors. Comparison of methodologies should focus on these different dimensions, and their relationship to the needs of the project or organization. Four issues concern the weight and use of methodologies: how how how how

development cost is affected by the form of communication, cost is affected by the weight of the methodology, methodology size relates to project size, and methodology specific density is justified by system criticality.

There is little research available, and much needed, on these issues. The bulkiness of methodologies is considered one of their main shortcomings. However, most of the bulkiness is contained in the technique descriptions. The technique descriptions can be used create special courses to teach the techniques, thus taking that aspect of the bulkiness out. The rest of the methodology need not be bulky, on a per person basis. That remainder names the reviews the person must attend, and the standards they must adhere to. Very few people should have to sit down and read the entire methodology. References [Cockburn94] Cockburn, A., "In Search of Methodology", Object Magazine, July, 1994, pp. .

[Cockburn98] Cockburn, A., Surviving Object-Oriented Projects, AddisonWesley, 1998. [Ehn93] Ehn, P., "Scandinavian Design: On Participation and Skill,", in Schuler, D., Namioka, A., eds., Participatory Design: Principles and Practice, Lawrence Erlbaum, inc., 1993, pp.41-77. [Goodwin84] Goodwin, C., "Notes on Story Structure and the organizaton of participation. In J.M.Atkinson, J. Heritage, eds, Structures of Social Action, Cambridge U. Press, 1984, pp 225-246. [Graham97] Graham, I., Henderson-Sellers, B., Younessi, H., The OPEN Process Specification, Addison-Wesley, 1997. [Harrison96] Harrison, N., Coplien, J, "Patterns of productive software organizations", Bell Labs Technical Journal, Summer, 1996, pp. 138-145. [Humphreys97] Humphreys, W., Introduction to the Personal Software Process, Addison-Wesley, 1997.

26 von 27

[Lave91] Lave, J., Wenger, E., Situated Learning: Legitimate Peripheral Participation, Cambridge U. Press, 1991. [Martin96] Martin, J., Odell, J., Object-oriented Methods, Pragmatic Considerations, Prentice Hall, 1996. [OOTC97] The IBM OOTC, Developing Object-Oriented Software, Wiley, 1997. [Plowman95] Plowman, L., "The interfunctionality of talk and text", CSCW, vol 3, 1995, pp.229-246. [RDD] Wirfs-Brock, R., McKean Alan, Responsibility-Driven Design, Tutorial #3, OOPSLA '97 [Sillince96] Sillince, J.A., "A model of social, emotional and symbolic aspects of computer-mediated communication within organizations", CSCW vol. 4, 1996, pp. 1-31. [XP] Jeffries, R., Beck, K., Extreme Programming, http://www.armaties.com /extreme.htm.

27 von 27