International Software Process Association (ISPA)

author(s):	Khaled El Emam, Barry Shostak, Nazim H. Madhavji
title:	Implementing Concepts from the Personal Software Process in an Industrial Setting
organisation(s):	Fraunhofer - Germany, CAE Ltd. Canada, McGill University, Centre de recherche informatique de Montreal, Canada

Implementing Concepts from the Personal
Software Process in an Industrial Setting

The Personal Software Process (PSP) is an evolutionary series of personal techniques that a software engineer learns and practices. In its initial incarnation, the PSP was intended as a classroom exercise course for graduate or senior-level undergraduates. PSP courses have been introduced in a number of universities in North America, such as Carnegie Mellon University, Embry-Riddle Aeronautical University, McGill University, Bradley University, and the University of Massachusetts. PSP courses have also been used to train professional engineers in industry. For example, PSP experiences have been reported in DEC, Motorola, Lawrence Livermore National Laboratory, and Union Switch and Signal. Experience thus far suggests that university students and professional programmers do benefit from PSP during the course.

From an industrial perspective, the effectiveness of PSP should be evaluated based on what engineers do in their real programming tasks (as opposed to course exercises). For example, the manner in which the PSP exercises are designed allows for substantial reuse from one exercise to the next. Reuse can potentially explain the improvements witnessed by students in terms of rising productivity and reductions in defect density. Furthermore, if students do not continue using their PSP skills after the course and if usage does not improve their real programming performance, then there is little motivation for management to support PSP training. As a precursor to our study, we interviewed four ex-students who had graduated from McGill University and who had taken the PSP course. They were currently programming in industry. None of them was applying the PSP concepts in their real programming tasks. Reasons cited for this included a lack of organizational commitment and support, and difficulty in applying the PSP concepts directly in their working environments.

We conducted a study on the implementation of some PSP concepts (namely personal measurement and structured code reviews) in a software development organization. The organization with whom our study was conducted has a population of approximately 1200 software engineers, and is well known for its flight simulators. We had 28 programmers who volunteered to take part in the study. The Human Resources Department in conjunction with the site SEPG decided on a measurement based process improvement strategy. The implementation of concepts from PSP was considered as part of this overall strategy.

It was decided that a pilot study would be conducted to: (a) tailor named PSP concepts to the context of the company, (b) support a climate for change within the organization (especially towards a measurement oriented culture), (c) evaluate the benefits of adopted PSP concepts within the company, and (d) evaluate the extent to which the implementation method has resulted in changes to the practices of the participating software engineers.

The approach that we chose for implementing the PSP concepts was to give the participants the lectures, and then let them apply the concepts that they have learned to real programming tasks on the job. After they had successfully applied the concepts in their programming tasks, they would be given another lecture where they are taught new concepts, which they would then be expected to apply to their real programming tasks on the job. This contrasts with the more commonly used approach of giving the students lectures over an extended period of time during which they apply the concepts to complete the textbook programming exercises, and afterwards they would apply the whole set of concepts to their real programming tasks.

Our empirical research method is action research. Following the action research method, researchers are involved directly in the introduction, observation, and evaluation of planned organizational change. Action research strives to achieve two objectives: (a) to solve practical organizational change problems, and (b) to increase our stock of scientific knowledge.

The implementation is described in terms of four phases: planning, training, evaluation, and leveraging. The problems and the manner in which they were addressed in each phase are presented.

Our results from the evaluation phase indicate that 57% of the software engineers who were taught the PSP concepts had the opportunity to apply the concepts in their real programming tasks. Also, 81% of the engineers who had the opportunity continue to apply the concepts in their programming tasks seven months afterwards. The engineers who applied code reviews witnessed substantial improvements in their defect removal capabilities. On average, participants more than doubled the amount of defects that they have found after code reviews, and more than halved the amount of time they spent on unit testing.

2 General Lessons Learnt

Some of the lessons that we have learned in this study are summarized below:

* Lectures should cover all the typical personal life cycles (e.g., a maintenance and a prototyping life cycle) that are in effect in the organization, not only the one presented in the PSP manuscript. This makes the classroom teaching more relevant to the participants' real programming tasks.

* It is important to customize the PSP data collection forms to the personal processes of individual participants. We found, for example, that personal processes varied depending on which product line the programmers were working on.

* All forms must be piloted with the participants in their real work environment. Even if the forms were designed to fit their personal processes, actual use in realistic programming tasks may reveal deficiencies in the design of the forms.

* It would be better to give the supervisors of the participants at least a formal short overview of PSP so that they understand it and see its benefits. This would help gain stronger commitment for PSP.

* It is important to have automated tools that support the participants' data collection and also data analysis. Furthermore, it would be preferable if the data collection forms are available in editable format for the participants so that they can customize the forms themselves as they gain a better understanding of their processes (e.g., to remove or add activities).

3 Conclusions

Our study on the implementation of concepts from PSP would support two conclusions. First, that the implementation of PSP should be approached as one would approach any technology implementation effort. This includes customizing PSP forms and data collection procedures to the organization, obtaining management committment and support, and conducting pilot and evaluation studies (i.e., just giving programmers PSP lectures and classroom exercises would not suffice). Second, that code reviews combined with unit testing increase the defect detection capability of programmers. We could not find evidence that programmer productivity has increased.

Table of Contents

Session 9 & 10