From Epowiki

Jump to: navigation, search

This page contains references to various papers, articles, and studies on code reviews/inspections.



The idea of this page is to collect information sources together so people can make up their own minds on code reviews. The literature is generally favorable to code reviews, so you won't find many dissenting voices, but the actual approach to code reviews is open to a lot of debate and is the subject of much research.

The following taxonomy of potential drivers of inspection costs and benefits.

  • structure (how the steps of the inspection are organized into a process);
  • techniques (how each step is carried out);
  • inputs (reviewer ability and code quality);
  • environment (interactions with other inspections, project schedule, personal calendars); and
  • technology (tool support).

Role of Meetings in Code Reviews

Should Computer Scientists Experiment More?

However, instinct and personal experience sometimes lead down the wrong path and computer science is no exception. Here are some examples. For about twenty years, it was thought that meetings were essential for software reviews. However, recently Porter and Johnson found that reviews without meetings are neither substantially more nor less effective than those with meetings. Meetingless reviews also cost less and cause fewer delays, which can lead to a more effective inspection process overall.

Identifying the Fundamental Drivers of Inspection Costs and Benifits

  • Structural factors had no significant effect on effectiveness
    • more reviewers didn’t always find more defects
  • Process inputs were far more influential than process structure
  • Best explanation of inspection effectiveness (so far)
    • not process structure
    • reviewer expertise
  • Meetingless inspections identified the most defects
    • also, generated the most issues and false positives
  • Few “meeting-sensitive” faults
  • Additional data
    • similar study at the University of Hawaii shows same results (Johnson97, Porter and Johnson97)
    • industrial case study of 3000 inspections showed that meetingless inspections were as effective as those with meetings (Perpich, Perry, Porter, Votta, and Wade97)
  • Best explanation of inspection effectiveness (so far)
    • not process structure nor group dynamics
    • reviewer expertise
  • Current models may be unfounded
    • meetings not necessarily cost-effective
    • more complex structures did not improve effectiveness
  • Reviewer expertise appears to be dominant factor in inspection effectiveness structure had little effect
    • inputs more influential than structure
    • individual effects more influential than group effects
    • improved individual analysis methods significantly improved performance

Assessing Software Review Meetings: Results of a Comparative Analysis of Two Experimental Studies

Abstract—Software review is a fundamental tool for software quality assurance. Nevertheless, there are significant controversies as to the most efficient and effective review method. One of the most important questions currently being debated is the utility of meetings. Although almost all industrial review methods are centered around the inspection meeting, recent findings call their value into question. In prior research the authors of this paper separately and independently conducted controlled experimental studies to explore this issue. This paper presents new research to understand the broader implications of these two studies. To do this, we designed and carried out a process of “reconciliation” in which we established a common framework for the comparison of the two experimental studies, reanalyzed the experimental data with respect to this common framework, and compared the results. Through this process we found many striking similarities between the results of the two studies, strengthening their individual conclusions. It also revealed interesting differences between the two experiments, suggesting important avenues for future research.

Asynchronous Software Inspection (1997)

Abstract: Although there exists a multitude of different inspection processes, the basic process has remained unchanged since it was first defined by Fagan in 1976. This process has as its central component an inspection meeting which all participants attend. But is this meeting cost effective? Recent empirical work has indicated this is not the case. An inspection model and prototype tool that dispense totally with the need for the inspectors to hold face-to-face meetings are presented.

A Review of Software Inspections (1995)

Abstract: For two decades, software inspections have proven effective for detecting defects in software. We have reviewed the different ways software inspections are done, created a taxonomy of inspection methods, and examined claims about the cost-effectiveness of different methods. We detect a disturbing pattern in the evaluation of inspection methods. Although there is universal agreement on the effectiveness of software inspection, their economics are uncertain.

A Controlled Experiment to Assess the Effectiveness of Inspection Meetings

Abstract: We hypothesize that inspection meetings are far less effective than many people believe and that meetingless inspections are equally effective. However, two of our previous industrial case studies contradict each other on this issue. Therefore, we are conducting a multi-trial, controlled experiment to assess the benefits of inspection meetings and to evaluate alternative procedures. The experiment manipulates four independent variables: (1) the inspection method used (two methods involve...

Reducing inspection interval in large-scale software development

We have found that, when software is developed by multiple, geographically separated teams, the cost-benefit trade-offs of software inspection change. In particular, this situation can significantly lengthen the inspection interval (calendar time needed to complete an inspection). Our research goal was to find a way to reduce the inspection interval without reducing inspection effectiveness. We believed that Internet technology offered some potential solutions, but we were not sure which technology to use nor what effects it would have on effectiveness. To conduct this research, we drew on the results of several empirical studies we had previously performed These results clarified the rote that meetings and individuals play in inspection effectiveness and interval. We conducted further studies showing that manual inspections without meetings were just as effective as manual inspections with them. On the basis of these and other findings and our understanding of Internet technology, we built an economical and effective tool that reduced the interval without reducing effectiveness. This tool, Hypercode, supports meetingless software inspections with geographically distributed reviewers. HyperCode is a platform independent tool, developed on top of an Internet browser, that integrates seamlessly into the current development process. By seamless, we mean the tool produces a paper flow that is almost identical to the current inspection process. HyperCode's acceptance by its user community has been excellent. Moreover, we estimate that using HyperCode has reduced the inspection interval by 20 to 25 percent. We believe that, had we focused solely on technology (without considering the information our studies had uncovered), we would have created a more complex, but not necessarily more effective tool. We probably would have supported group meetings, restricted each participant's access to review comments and supported a wider variety of inspection methods. In other words, the principles derived from our empirical studies dramatically and successfully directed our search for a technological solution.

An Experiment to Assess Cost-Benefits of Inspection Meetings and their Alternatives: A Pilot Study (1996)

Anywhere Anytime Code Inspections Using the Web to Remove Inspection Bottlenecks in Large Scale Software Development

Disseminating information maintaining artifact consistency and scheduling coordinated activities are critical problems in any large scale software development Inadequate management of this process overhead can increase rework e ort decrease quality and lengthen interval These problems are greatly magnified when a development team is divided across two or more geographically separate locations.

A Case Study of Distributed, Asynchronous Software Inspection

Assessing Software Review Meetings: A Controlled Experimental Study Using CSRS

Shared Defect Detection : The Effects of Annotations in Asynchronous Software Inspection

Software development is constrained by time, cost and quality requirements. Software defects are one of the most influential factors affecting the success of a software project. Software inspection has become an essential solution for managing software defects. A variety of software inspection techniques have been developed. Traditionally, software inspections are performed manually and require meeting-based activities. New inspection techniques have challenged the conventions by eliminating the defect collection meeting. The original asynchronous software inspection consists of private and public reviews, and it has been proven as effective as traditional ones. Different kinds of computer technologies are used to augment the software inspection process. This study empirically examined the effects of annotations of shared defect detection in asynchronous software inspection in which the private review is absent and all annotations are visible to members of the inspection team at all times. A Latin square experimental design was employed, and twenty-seven novices participated in the experiment as inspector subjects. The study results show that the defect discovery ratio is not affected by the number of annotations in the target materials under inspection. Thus, the study suggests that defect detection task and defect review task can be combined without loss. This basic finding of this research supports the proposition that more flexible focus of software inspection (i.e. asynchronous and shared annotation) may be adopted without negative impacts on defect detection. The analysis study also addresses additional issues related to software inspection, augmentation, cognition, and research design.

Towards a Problem-Driven Approach to Perspective-Based Reading (2002)

Abstract: The quality of a requirements specification has a great impact on the quality of the software developed. Because of this, a requirements specification should be complete, correct, consistent, and unambiguous. Otherwise, defects may remain undetected, resulting in the delivery of a faulty software product to the users. Motivated by this, Basili et al. have developed the perspective-based reading (PBR) technique to help identify defects in requirements specifications. In this paper, we propose a...

Does every inspection need a meeting?

Comparing detection methods for software requirements inspections

Abstract: Software requirements specifications (SRS) are often validated manually. One such process is inspection, in which several reviewers independently analyze all or part of the specification and search for faults. These faults are then collected at a meeting of the reviewers and author(s). Usually, reviewers use Ad Hoc or Checklist methods to uncover faults. These methods force all reviewers to rely on nonsystematic techniques to search for a wide variety of faults. We hypothesize that a...

Code Reviews and Product Quality

Design and code inspections to reduce errors in program development

Substantial net improvements in programming quality and productivity have been obtained through the use of formal inspections of design and of code. Improvements are made possible by a systematic and eficient design and code verijicution process, with well-dejined roles for inspection participants. The manner in which inspection data is categorized and made suitable for process analysis is an important factor in attaining the improvements. It is shown that by using inspection results, a mechanism for initial error reduction followed by ever-improving error rates can be achieved.

Evaluation of Code Review Methods through Interviews and Experimentation

Abstract: This paper presents the results of a study where the effects of introducing code reviews in an organisational unit have been evaluated. The study was performed in an ongoing commercial project, mainly through interviews with developers and an experiment where the effects of introducing code reviews were measured. Two different checklist based review methods have been evaluated. The objectives of the study are to analyze the effects of introducing code reviews in the organizational unit.

An Empirical Study of Inspection and Testing

This paper presents a case study of inspections and testing, done at the Ericsson development department outside Oslo in Norway. This department develops and maintains customer-defined services around AXE phone switches, i.e. the functionality around the "star" and "square" buttons on house telephones.

Software Analysis and Model Checking

Abstract: Most software developers today rely on only a small number of techniques to check their code for defects: peer review, code walkthroughs, and testing. Despite a rich literature on these subjects, the results often leave much to be desired. The current software testing process consumes a significant fraction of the overall resources in industrial software development, yet it cannot promise zero-defect code. There is reason to hope that the process can be improved.

Evaluating Defect Detection Techniques for Software Requirements Inspections (2000)

Abstract: Perspective-Based Reading (PBR) is a family of defect detection techniques which have been proposed to improve the effectiveness of software requirements inspections. PBR drives individual document reading by means of perspectivedependent procedural scenarios, which are different for each inspector in the team. Based on the former PBR experiments, we present a controlled experiment with more than one hundred undergraduate students who conducted software inspections.

A Survey of Software Inspection Technologies

Abstract: Software inspection is a proven method that enables the detection and removal of defects in software artifacts as soon as these artifacts are created. It usually involves activities in which a team of qualified personnel determines whether the created artifact is of sufficient quality. Detected quality deficiencies are subsequently corrected. In this way an inspection can not only contribute towards software quality improvement, but also lead to significant budget and time benefits.

An Experiment to Assess the Cost-Benefits of Code Inspections in Large Scale Software Development (1995)

Abstract: We conducted a long-term experiment to compare the costs and benefits of several different software inspection methods. These methods were applied by professional developers to a commercial software product they were creating. Because the laboratory for this experiment was a live development effort, we took special care to minimize cost and risk to the project, while maximizing our ability to gather useful data.

Technical reviews

Abstract: Reviews of software work products were introduced by Fagan in the 1970s. A review can be applied to any kind of software work product. A Fagan-type inspection is carried out in a meeting of four or five people who use checklists to help them locate problems in the product under review. Lately the effectiveness of both inspection meetings and checklists has been questioned. We examine Fagan-type inspections as well as alternatives to meetings and checklists.

An Encompassing Life-Cycle Centric Survey of Software Inspection (1998)

Abstract: This paper contributes an integrated survey of the work in the area of software inspection. It consists of two main sections. The first introduces a detailed description of the core concepts and relationships that together define the field of software inspection. The second elaborates a taxonomy that uses a generic development life-cycle to contextualize software inspection in detail. After Fagan's seminal work presented in 1976, the body of work in software inspection has greatly increased...

Identifying the Mechanisms to Improve Code Inspection Costs and Benefits

Software inspections have long been considered to be an effective way to detect and remove defects from software. However, there are costs associated with carrying out inspections and these costs may outweigh the expected benefits.

Experience with Fagan's Inspection Method (1992)

Abstract: this paper. In general, they have proved to be favorable. Because the costs of fixing errors in software were known, the payback for every hour invested in inspection was shown to be a factor 30. There are also other benefits that are much more difficult to quantify directly but whose effect is significant in terms of the overall quality of the software.

INSPECTIONS Evolution and History: 1972 � 2001

Identifying the Mechanisms to Improve Code Inspection Costs and Benefits

Software inspections have long been considered to be an effective way to detect and remove defects from software. However, there are costs associated with carrying out inspections and these costs may outweigh the expected benefits.

It is important to understand the tradeoffs between these costs and benefits. We believe that these are driven by several mechanisms, both internal and external to the inspection process. Internal factors are associated with the manner in which the steps of the inspection are organized into a process (structure), as well as the manner in which each step is carried out (technique). External ones include differences in reviewer ability and code quality (inputs), and interactions with other inspections, the project schedule, personal calendars, etc. (environment).

Many people believe that most defects are identified during the inspection meeting. However, several recent studies have indicated that most defects are actually found during the preparation step 55 62. Humphrey 32 states that "three-quarters of the errors found in well-run inspections are found during preparation." Votta 62 suggests replacing inspection meetings with depositions, where the author and, optionally, the moderator meet separately with each of the reviewers to get their inspection results.

Software Inspections are not for quality, but for engineering economics

The bottom line is that I believe that it is more relevant to view Inspection as a way to control the economic aspects of software engineering, rather than a way to get 'quality' by early defect removal. Quality needs to be multidimensional, specified in quantified requirements and architected, engineered and designed into software products. Inspection needs to be used to monitor the entire software engineering process.

Optimizing Software Inspections

Software Inspections are now widely known to the software industry, but most organizations do not make the most of them. This is because many people misunderstand and misinterpret Inspection. This article aims to provide some direction as to how to get the most value from your Inspections.

Generalizing Perspective-based Inspection to handle Object-Oriented Development Artifacts

The value of software inspection for uncovering defects early in the development lifecycle has been well documented. Of the various types of inspection methods published to date, experiments have shown perspective-based inspection to be one of the most effective, because of its enhanced coverage of the defect space. However, inspections in general, and perspective-based inspections in particular, have so far been applied predominantly in the context of conventional structured development methods, and then almost always to textual artifacts, such as requirements documents or code modules.

Testing, Reviews, and Inspections

Background and Theory of Inspections II. In-class Inspection of Instructor�s Document I. Group Inspection of Document Provided by One Member

The Old Concepts of Inspection

Review Techniques

Covers issues like perspective based reviews, check lists, uses cases, etc.

Are the Perspectives Really Different? Further Experimentation on Scenario-Based Reading of Requirements (2000)

Abstract: Perspective-Based Reading (PBR) is a scenario-based inspection technique where several reviewers read a document from different perspectives (e.g. user, designer, tester). The reading is made according to a special scenario, specific for each perspective. The basic assumption behind PBR is that the perspectives find different defects and a combination of several perspectives detects more defects compared to the same amount of reading with a single perspective. This paper presents a study which...

Perspetive Based Usability Inspection: An Emperical Validation of Efficacy

Abstract: Inspection fundamental means achieving software usability.Past researchshowed current usability inspection techniques were rather ineffective. We developed perspective-based usability inspection, which divides the large variety usability issues along different perspectives focuses each inspection session perspective. conducted a controlled experiment to study effectiveness, using post- only control group experimental design, with professionals subjects. The control group used heuristic...

The Empirical Investigation of Perspective-Based Reading (1996)

Abstract: We consider reading techniques a fundamental means of achieving high quality software. Due to the lack of research in this area, we are experimenting with the application and comparison of various reading techniques. This paper deals with our experiences with a family of reading techniques known as Perspective-Based Reading (PBR), and its application to requirements documents. The goal of PBR is to provide operational scenarios where members of a review team read a document from a particular...

An Empirical Study Of Perspective-Based Usability Inspection (1998)

Abstract: Inspection is a fundamental means of achieving software usability. Past researchshowed that during usability inspection the success rate (percentage of problems detected) of each individual inspector was rather low. We developed perspective-based usability inspection, which divides the large variety of usability issues along different perspectives and focuses each inspection session on one perspective. We conducted a controlled experiment to study its effectiveness, using a post-test only...

Perspective-based Usability Inspection: An Empirical Validation of Efficacy

Abstract: Inspection is a fundamental means of achieving software usability.Past researchshowed that the current usability inspection techniques were rather ineffective. We developed perspective-based usability inspection, which divides the large variety of usability issues along different perspectives and focuses each inspection session on one perspective. We conducted a controlled experiment to study its effectiveness, using a posttest only control group experimental design, with 24 professionals as...

An Internally Replicated Quasi-Experimental Comparison of Checklist and Perspective-based Reading of Code Documents (1999)

Abstract: The basic premise of software inspections is that they detect and remove defects before they propagate to subsequent development phases where their detection and correction cost escalates. To exploit their full potential, software inspections must call for a close and strict examination of the inspected artefact. For this, reading techniques for defect detection may be helpful since these techniques tell inspection participants what to look for and, more importantly, how to scrutinise a...

An Experimental Comparison of Reading Techniques for Defect Detection in UML Design Documents (1999)

Abstract: The basic motivation for software inspections is to detect and remove defects before they propagate to subsequent development phases where their detection and removal becomes more expensive. To maximize this potential, the examination of the artefact under inspection must be as thorough and detailed as possible. This implies the need for systematic reading techniques that tell inspection participants what to look for and, more importantly, how to scrutinise a software document. Recent...

Perspective-based Reading of Code Documents at Robert Bosch GmbH (1997)

Abstract: Despite dramatic changes in software development in the two decades since the term software engineering was coined, software quality deficiencies and cost overruns continue to afflict the software industry. Inspections, developed at IBM by Fagan in the early 1970s, can be used to improve upon these problems because they allow the detection and removal of defects after each phase of the software development process. But, in most published inspection processes, individuals performing defect...

An Experimental Comparison of Checklist-Based Reading and Perspective-Based Reading for UML Design Document Inspection

Abstract: This paper describes an experimental comparison of two reading techniques, namely Checklist-based reading (CBR) and Perspective-based reading (PBR) for Object-Oriented (00) design inspection. Software inspection is an effective approach to detect defects' in the early stages of the software development process. However, inspections are usually applied for defect detection in software requirement documents' or software code modules, and there is a significant lack of information how inspections...

Precise Use Cases

This paper describes a precise form of use case thatpromotes the specification of actor options and alternativecourse conditions. Precise cases use a precise activitydescription language with a Structured English grammar.The design goals for the notation were to maximizeunderstandability by application experts, not familiar withformal object-oriented concepts, and to supply sufficientinformation and structure to support both manual andautomated test design. In addition to a detailed example,a development process, and guidelines for modeling andtest coverage are provided.

An Experimental Comparison of Usage-Based and Checklist-Based Reading

Software quality can be defined as the customer’s per-ception of how a system works. Inspection is a method tocontrol the quality throughout the development cycle. Read-ing techniques applied to inspections help reviewers to stayfocused on the important parts of an artefact when inspect-ing. However, many reading techniques focus on finding asmany faults as possible, regardless of their importance. Us-age-based reading helps reviewers to focus on the most im-portant part of an software artefact from a user’s point ofview. This paper is an extended abstract of a technical re-port describing an experiment, which compares usage-based and checklist-based reading. The results show thatreviewers applying usage-based reading are more efficientand effective in detecting the most critical faults from a us-er’s point of view than reviewers using checklist-basedreading. Usage-based reading may be preferable to use forsoftware organisations utilising or will start utilising usecases in their software development.

Combining Scenario-based Requirements withStatic Verification and Dynamic Testing

Abstract.Two important prerequisites for achieving high quality software are solid requirements engineering and systematic testing. Scenarios and usecases are gaining increased attention in requirements engineering, as meansfor eliciting, documenting and validating requirements. Scenarios may alsobe a basis for testing. This paper identifies a number of possibilities of com-bining scenario-based requirements engineering with systematic approachesfor software verification and validation. It presents outlines of strategies forutilizing scenarios created during requirements engineering as input to staticverification and dynamic testing. Special attention is given to the opportunityof (1) using prioritized scenarios for guiding inspections, (2) combining sce-narios with reliability testing, and (3) scenario elicitation through strategiescommon in testing. Several issues of further research are identified.

Empirical Investigation of the Success Factors of Scenario Based Reading

Active Design Reviews

The Darker Side of Inspection

The purpose of inspections is to intensify the quality of software development. In simple terms, the goal is to find errors in any sort of software development document. Although unlike testing, inspections are performed by groups of human beings � not computers � they are usually regarded as technical manoeuvres and the human or sociological aspect is forgotten.


The Logic of Bugs

Abstract: Real-life bugs are successful because of their unfailing ability to adapt. In particular this applies to their ability to adapt to strategies that are meant to eradicate them as a species. Software bugs have some of these same traits. We will discuss these traits, and consider what we can do about them.

Reading Techniques for OO Design Inspections (2002)

Abstract: Inspections can be used to identify defects in software artifacts. In this way, inspection methods help to improve software quality, especially when used early in software development. Inspections of software design can be especially crucial since design defects (problems of correctness and completeness with respect to the requirements, internal consistency, or other quality attributes) can directly affect the quality of, and effort required for, the implementation.

A Method for Evaluation of the Performance of Requirements Inspections (2000)

Abstract: A new improved model is introduced for the performance of inspectors of user requirements documents. The model, which is based on a linearity assumption employs only two non-dimensional metrics in the 0;1 interval. They characterise the error detection ability of the inspectors and their domain knowledge level. It is our experience that one learns to reason with these two metrics. The application of the model is demonstrated with an analysis of which kind of inspectors.

Should you test the code before you test the program? (2001)

Abstract: Suppose a software development manager is considering whether or not to add automatic code inspection to the current development process. The paper shows what properties of the current process to measure, and how to use these measurements in order to make a rational decision. Let P1 denote the process whose steps are {write code, test program against requirements, find faults, fix faults} and P2 be the process P1 augmented by the single step {test code against guidelines}.

Looking for inspection improvements through the base practices

Abstract: Inspection process improvement is typically a non-rational process, in which the improvement suggestions come from randomly arranged brainstorming meetings. The improvement ideas focus mainly on updating of checklists, adoption of new forms and definition of new metrics for measurement data. These are important and usually the most relevant improvement topics, but we strive for a more rigid improvement process, which pays attention to case-specific improvement actions of the inspection process.

Monitoring Software Inspections With Prescriptive Metrics (1997)

Abstract: This paper therefore focuses on inspection, its evaluation and improvement, and presents a set of prescriptive software metrics for evaluating software inspections. The usability of the metrics is further illustrated with a case study involving a Finnish software company.

Experiences of Painless Improvements in Software Inspection

Abstract: An inspection is regarded by software developers as an efficient method for finding errors. Unfortunately companies do not necessarily have the power to implement inspection efficiently. Why we have started a project in order to find a suitable way to carry out inspections in four companies.

The Web Generation of Software Inspection: A Process with Virtual Meetings and On-line Recording

Abstract: Face-to-face meetings typically provide a natural way of negotiating and collecting opinions. The problem is that these meetings also cause resource problems, i.e. they waste time and are difficult to arrange. A shift to more flexible implementations of meetings is thus an understandable alternative. We could then remove barriers to the inclusion of inspections in the development process. The present paper outlines a solution to this problem and introduces a web generation of inspections.

The virtual logging meeting: a web-based solution to resource problems in software inspection (1999)

Abstract: Introduction It is largely accepted that formal inspection is the most effective means of finding defects, so that it plays an important role in the software development process. Preparation for the inspection (individual inspection) and the logging meeting form the major phases in the conventional inspection process. Face-to-face meetings typically provide a natural way of negotiating and collecting opinions. The problem is that these meetings also cause resource problems.

Inspecting Postscript Documents In An Object-Oriented Environment

Abstract: this paper we focus on rules and checklists in particular, and on some problems that have been discovered. The major problem concerning rules and checklists arises from their generality, i.e. checklists are not tailored to a specific development method, development phase, development principle in a company or development principle in a specific project. This insufficient tailoring is used as an excuse when justifying the lack of checklists in software inspection.

Scaling up Distributed Software Inspections (2001)

Abstract: Introduction Software inspections are a software engineering "best practice" for detecting and correcting defects in software artifacts, including requirements, design, code and test cases. Because defects are detected early in the software development process, software inspection improves software quality and reduces rework, thus saving time and costs. As a software process, inspections are adopted in many variants 6 by organizations such as Ericsson, Hewlett-Packard, Lucent Technologies.

Distributed and Colocated Projects: a Comparison

Abstract: The aim of this work is to point out through experimentation some of the problems that arise with distributed software development, such as the need for new techniques and methods for managing projects and processes, so as to achieve better assignment of activities among the various working groups, together with efficient communication among the members of each team. The paper analyzes the post mortem data on two projects, one conducted at a single site and the other at several different sites..

Evaluating the Usefulness and the Ease of Use of a Web-based Inspection Data Collection Tool (1998)

Abstract: This paper contributes a valid and reliable measurement instrument in the form of a questionnaire to determine, from a user's perspective, the usefulness and ease of use of a Web-based Inspection Process Support tool (WIPS) that we developed for inspection data collection. The questionnaire is built upon the work of Fred Davis on perceived usefulness, ease of use, and usage of information technology. To validate the questionnaire and its underlying model as well as to evaluate WIPS

Statistical Analysis of Two Experimental Studies (1998)

Abstract: Software inspection is well-known as an effective defect finding process. One avenue of inspection research concerns tool support, with the aim of further increasing its efficiency and effectiveness. This has resulted in the production of a number of prototype tools. In general, however, these tools are poorly evaluated and the performance gain which these tools may provide has not been quantified. This paper describes an incremental, cyclic approach to the development and evaluation of a tool.

A Comparison of Tool-Based and Paper-Based Software Inspection (1998)

Abstract: Software inspection is an effective method of defect detection. Recent research activity has considered the development of tool support to further increase the efficiency and effectiveness of inspection, resulting in a number of prototype tools being developed. However, no comprehensive evaluations of these tools have been carried out to determine their effectiveness in comparison with traditional paper-based inspection.

Further Experiences with Scenarios and Checklists (1998)

Abstract: Software inspection is one of the best methods of verifying software documents. Software inspection is a complex process, with many possible variations, most of which have received little or no evaluation. This paper reports on the evaluation of one component of the inspection process, detection aids, specifically using Scenario or Checklist approaches. The evaluation is by subject-based experimentation, and is currently one of three independent experiments on the same hypothesis.

Automated Generic Support for Software Inspection (1997)

Abstract: Software inspection is a widely used method for finding defects in all types of software development documents. Many process variations exist, each designed for use under certain circumstances or to address some perceived fault in existing methods. A desirable attribute of inspection is rigour, allowing the use of historical data to predict future performance and suggest process improvements. Recent work in tool support for inspection is designed to tackle the issue of enforcing rigorous...

An Empirical Study of Web-Based Inspection Meetings

Software inspections are a software engineering “best practice” for defect detection and rework reduction. In this paper, we describe an empirical evaluation with using a tool aiming to provide Internet groupware support for distributed software inspections. The tool is based on a restructured inspection process where inspection meetings have the only goal of removing false positives rather than finding additional defects. In place of face-to-face meetings, the tool provides web-based discussion forums and support for voting

We present an empirical study of nine remote inspections which were held as part of a university course. We investigated whether all collected defects are worth to be discussed as a group. Results show that discussions for filtering out false positives (non true defects) might be restricted to defects which were discovered by only one inspector

Investigating Reading Techniques for Object-Oriented Framework Learning

Abstract: The empirical study described in this paper addresses software reading for construction: how application developers obtain an understanding of a software artifact for use in new system development. This study focuses on the processes that developers would engage in when learning and using object-oriented frameworks. We analyzed 15 student software development projects using both qualitative and quantitative methods to gain insight into what processes occurred during framework usage. The...

Lessons learned from 25 years of process improvement: The Rise and Fall of the NASA Software Engineering Laboratory (2002)

Abstract: For 25 years the NASA/GSFC Software Engineering Laboratory (SEL) has been a major resource in software process improvement activities. But due to a changing climate at NASA, agency reorganization, and budget cuts, the SEL has lost much of its impact. In this paper we describe the history of the SEL and give some lessons learned on what we did right, what we did wrong, and what others can learn from our experiences. We briefly describe the research that was conducted by the SEL, describe how we...

Computer Supported Software Inspection

Abstract: For over twenty years, software inspection has been advocated as a simple, cost-effective technique for defect detection in all types of documents. Essentially, a number of participants review a document with the aim of discovering defects. Many positive experience reports have been published demonstrating its benefits, and there are now a number of variations on the basic inspection method. In recent years, there have been a number of attempts to further increase inspection efficiency by the...

Experimental Evaluation of Program Slicing for Fault Localization (2002)

Abstract: Debugging large and complex software systems requires significant effort since it is very difficult to localize and identify faults. Program slicing has been proposed to efficiently localize faults in the program. Despite the fact that a number of debug systems using program slicing, have been developed, the usefulness of this method to fault localization has not been sufficiently evaluated. This paper aims to experimentally evaluate the usefulness of the program slicing method to fault...

An Empirical Methodology for Introducing Software Processes (2001)

Abstract: There is a growing interest in empirical study in software engineering, both for validating mature technologies and for guiding improvements of less-mature technologies. This paper introduces an empirical methodology, based on experiences garnered over more than two decades of work by the Empirical Software Engineering Group at the University of Maryland and related organizations, for taking a newly proposed improvement to development processes from the conceptual phase through transfer to...

Software Inspection Benchmarking - A Qualitative and Quantitative Comparative Opportunity

Software inspections are commonly used throughout the software industry, but there are still open questions about inspection effectiveness. For example, which techniques work best in various environments? Are requirements specifications inspections and code inspections different in terms of effectiveness? What is the effectiveness in inspections for different group sizes? Benchmarking provides an opportunity to address such issues. This paper discusses how benchmarking may be applied for software inspections. The discussion is illustrated with an empirical study. It is shown how the data can be used to plan and manage software inspections. It is concluded that software inspections are well suited for benchmarking and that software practitioners as well as researchers can learn valuable lessons. the relationship between inspection processes and

Hyper Code

Software Inspection and Computer Support

The Web Generation of Software Inspection: A Process with Virtual Meetings and On-line Recording

Abstract: Face-to-face meetings typically provide a natural way of negotiating and collecting opinions. The problem is that these meetings also cause resource problems, i.e. they waste time and are difficult to arrange. A shift to more flexible implementations of meetings is thus an understandable alternative. We could then remove barriers to the inclusion of inspections in the development process. The present paper outlines a solution to this problem and introduces a web generation of inspections, which ...

Comprehension and Visualisation of Object-Oriented Code for Inspections

Abstract: This paper considers the role of comprehension during the preparation and defect detection phases of the software inspection process. Software inspection is generally accepted as a useful technique for finding errors in both documents and code. However, there is no general agreement on how defects are best detected and, in particular, how much understanding of the product is required and how that understanding is best achieved. Some inspection processes provide no guidance. Many advocate fairly informal aids such as checklists. Recently more structured techniques, in the form of scenarios, have been proposed. The need for increased comprehension seems particularly relevant to object-oriented technology as a result of inherent features, which appear to increase inter-component dependencies. This paper reviews the Software Engineering literature investigating the role of comprehension and the related topic of program visualisation during the preparation and defect detection phases of inspection. It considers particular features of object-oriented technology that may require enhanced comprehension during inspection. It draws on similarities between software maintenance and software inspection to suggest that there are potential benefits to be obtained in using comprehension techniques and tools, developed for maintenance, during inspection.

Scrutiny: A Collaborative Inspection and Review System

Abstract: This paper describes a Bull US Applied Research Laboratory project to build a collaborative inspection and review system called Scrutiny using Conversation Builder from the University of Illinois at Urbana-Champaign. The project has several distinct aspects: technology oriented research, prototype building, experimentation and tools deployment/technology transfer. Described are the design of the current operational version of Scrutiny for inspection-only, the evolutionary design of Scrutiny to handle various forms of review, and some initial thoughts on integration with other CASE frameworks and tools. The problem domain selected, the development environment, lessons learned thus far, some ideas from related work, and the problems anticipated are discussed.

Reengineering Inspection: The Future of Formal Technical Review

Abstract: Formal technical review is acknowledged as a preeminant software quality improvement method. The "inspection" review method, first introduced by Michael Fagan twenty years ago, has led to dramatic improvements in software quality. It has also led to a myopia within the review community, which tends to view inspection-based methods as not just effective, but as the optimal approach to formal technical review. This article challenges this view by presenting a taxonomy of software review that shows inspection to be just one among many valid approaches. The article then builds upon this framework to propose seven guidelines for the radical redesign and improvement of formal technical review during the next twenty years.

Experiences with CSRS: An Instrumented Software Review Environment

Abstract: Formal technical review (FTR) is a cornerstone of software quality assurance. However, the labor-intensive and manual nature of review, along with basic unresolved questions about its process and products, means that review is typically under-utilized or inefficiently applied within the software development process. This paper discusses our initial experiments using CSRS, an instrumented, computer-supported cooperative work environment for software review that reduces the manual, labor-intensive nature of review activities and supports quantitative study of the process and products of review. Our results indicate that CSRS increases both the breadth and depth of information captured per person-hour of review time, and that its design captures interesting measures of review process, products, and effort.

Supporting Technology Transfer of Formal Technical Review Through a Computer Supported Collaborative Review System

Abstract: Formal technical review (FTR) is an essential component of all modern software quality assessment, assurance, and improvement techniques, and is acknowledged to be the most cost-effective form of quality improvement when practiced effectively. However, traditional FTR methods such as inspection are very diffcult to adopt in organizations: they introduce substantial new up-front costs, training, overhead, and group process obstacles. Sustained commitment from high-level management along with substantial resources is often necessary for successful technology transfer of FTR. Since 1991, we have been designing and evaluating a series of versions of a system called CSRS: an instrumented, computer-supported cooperative work environment for formal technical review. The current version of CSRS includes an FTR method definition language, which allows organizations to design their own FTR method, and to evolve it over time. This paper describes how our approach to computer supported FTR can address some of the issues in technology transfer of FTR.

Reading Computer Programs

Abstract: The ability to read and understand a computer program is a criti-cal skill for the software developer, yet this skill is seldom developed in any systematic way in the education or training of software professionals. These materials discuss the importance of program reading, and review what is known about reading strategies and other factors affecting comprehension. These materials also include reading exercises for a modest Ada program and discuss how educators can structure additional exercises to enhance program reading skills.

Predictive Quality Control with Software Inspections

Abstract: Over the past decade, software inspections have established an impressive track record for early defect detection and correction, and their use has grown extensively. But during this time software quality requirements have also increased, raising the level of our quality expectations. Industrial engineering has addressed similar increasing quality requirements for hardware by being able to predict problems early in development by using statistical process control (SPC). Applying SPC to software inspection data provides a similar ability to predict and remove latent problems, those that haven't been identified as yet, early in software development before they can propagate and cause later difficulty and expense. The application of SPC to software inspection data to predict defects is defined and illustrated using sample inspection data, and recommendations are made for applying predictive quality control

Hyper Text representation of program source Code

In this paper we introduce Hyper Code, a Hyper Text representation of program source code. Using HTML for code presentation, Hyper Code provides links from uses of functions, types, variables, and macros to their respective definition sites; similarly, definitions are linked to lists-of-links back to use sites. Standard HTML browsers such as Mosaic thereby become powerful tools for understanding program control flow, functional dependencies, data structures, and macro and variable utilization. Supporting Hyper Code with a code database front-ended by a WWW server enables software sharing and development on a global scale by leveraging the programming, debugging, and computing power brought together by the World-Wide Web.

Virtual Software Inspections over the Internet

Software inspections, although they have proved cost-effective, are still practised infrequently or not at all. Their value may be understated and the effort required for arranging inspection meetings is too often considered to be a waste of time.

Anywhere, anytime code inspections:using the web to remove inspection bottlenecks in large-scale software development

Disseminating information, maintaining artifact consis- tency, and scheduling coordinated activities are critical problems in any large-scale, software development. In- adequate management of this "process overhead" can increase rework e�ort, decrease quality, and lengthen interval. These problems are greatly magni�ed when a development team is divided across two or more ge- ographcally separate locations. tional inspection with meetings. Keywords Code inspections: web-based, meetingless, asyn- chronous; Natural occurring inspection experiment; Au- tomated support for inspections; Work, paper and in- formation ow.

Using Measurement-Driven Modeling to Provide Empirical Feedback to Software Developers

Several authors have explored the application of classification methods to software development. These studies have concentrated on identifying modules that are difficult to develop or that have high fault density. While this information is important it provides little help in determining appropriate corrective action. This article extends previous work by applying one classification method, classification tree analysis (CTA), to more a fine-grained problem routinely encountered by developers. In this article we use CTA to identify software modules that have specific types of faults (i.e Logic, interface, etc.) We evaluate this approach using data collected from six actual software projects. Overall, CTA was able to correctly differentiate faulty modules from fault-free modules in 72% of the cases. Furthermore, 82% of the faulty modules were correctly identified. We also show that CTA outperformed two simpler classification strategies.

Virtual Software Inspections for Distributed Software Engineering Projects

ABSTRACT Traditional software inspection is recognized method, but is poorly suited to modern distributed software engineering projects. Arranging inspections in a distributed and asynchronous manner is not a straightforward matter. Over and over again, researchers have constructed tools with unique sets of features to support paper-based inspections. Now the concept of virtual software inspection is introduced to overcome that problem. This is based on three important aspects � computer tools, flexibility and integration � and can be used as a reference model in distributed software projects. This paper briefly describes the concept, the tool supporting it, and ongoing experiments to evaluate these.

How Much Code Inspection Is Enough?

Given code inspection effectiveness (defects found during inspection/defects found during inspection plus those found during test) as a function of preparation rate (amount of code examined per labor hour), it is then possible to construct a simple cost model that predicts testing labor hours as a function of code inspection preparation rate. This paper develops that model and computes the optimum code inspection preparation rate to minimize total cost (inspection + test). Existing program data (with significant caveats) have been used together with certain rough approximations to show that Fagan-style code inspections obey a simple predictive cost model.

Tool Support for Distributed Inspection

Software inspection is one of the best practices for detecting and removing defects early in the software development process. We present a tool to support geographically distributed inspection teams. The tool adopts a reengineered inspection process to minimize synchronous activities and coordination problems, and a lightweight architecture to maximize easy of use and deployment.

Inspection State of the Art and Practice

Is Software Inspection Value Added?

Applying Program Comprehension Techniques to Improve Software Inspections

Abstract: Software inspections are widely regarded as a cost-effective mechanism for removing defects in software, though performing them does not always reduce the number of customer-discovered defects. We present a case study in which an attempt was made to reduce such defects through inspection training that introduced program comprehension ideas. The training was designed to address the problem of understanding the artifact being reviewed, as well as other perceived deficiencies of the inspection process itself. Measures, both formal and informal, suggest that explicit training in program understanding may improve inspection effectiveness.

Preliminary Evaluation of Tool-based Support for Distributed Inspection

Software inspections are a software engineering �best practice� for defect detection and rework reduction. In this paper, we describe our initial empirical evaluation with using a tool aiming to provide Internet groupware support for distributed software inspections. The tool is based on a restructured inspection process to reduce synchronization and coordination problems.

Managing the Software Process

#32 Watts S. Humphrey. Managing the Software Process, chapter 10. Addison-Wesley Publishing Company, 1989.

Lessons from 342 Medical Device Failures

Most complex systems today contain software, and systems failures activated by software faults can provide lessons for software development practices and software quality assurance. This paper presents an analysis of softwarerelated failures of medical devices that caused no death or injury but led to recalls by the manufacturers. The analysis categorizes the failures by their symptoms and faults, and discusses methods of preventing and detecting faults in each category. The nature of the faults provides lessons

about the value of generally accepted quality practices for prevention and detection methods applied prior to system release. It also provides some insight into the need for formal requirements specification and for improved testing of complex hardware-software systems.

Groupware support for asynchronous document review

The formal document review process has basically remained unchanged since it was first introduced by Fagan in 1976. Central to this process is the review meeting, which all participants attend. However, recent empirical work has indicated that this meeting is not necessarily effective. A review model and a prototype co-operative work tool, InspectA, that dispense totally with the need for reviewers to hold face-toface meetings, are presented. They replace the meeting with further individual reviews combined with communication between reviewers facilitated by electronic mail. The model and tool are both compared with other existing work in the area of meetingless review.

Reengineering Inspection

A 20-year-old inspection technique has served developers well in the quest for software quality improvement. But radical changes are on the horizon that may seriously steer the future of formal review.

Overcoming the NAH syndrome for inspection deployment

Despite considerable evidence to show that inspections can help reduce costs and improve quality, inspections are not widely deployed in the software industry. One of the likely reasons for this is the �not applicable here (NAH)� syndrome - developers and managers believe that in their environment, inspections will not provide the benefits seen by other organizations.

Understanding the sources of variation in software inspections

In a previous experiment, we determined how various changes in three structural elements of the software inspection process (team size and the number and sequencing of sessions) altered effectiveness and interval. Our results showed that such changes did not significantly influence the defect detection rate, but that certain combinations of changes dramatically increased the inspection interval. We also observed a large amount of unexplained variance in the data, indicating that other facto ...

CAIS: collaborative asynchronous inspection of software

Many software engineering tasks have a synchronous component that requires the participants to assemble together at the same time and place. This approach is expensive in terms of traveling, scheduling and human resources. Existing computer tools mitigate these constraints by adding structure to the meeting, providing on-line document support, and distributing the participants over geographic boundaries. The constraint remains, however, that all participants participate at the same time

Personal tools