International Software Process Association (ISPA)

author(s):	Christophe Debou, Annie Kuntzmann-Combelles
title:	From Business Goals to Improvement Planning: Practical Use of ami* in Industry
organisation(s):	Alcatel Telecom, Objectif Technologie
copyright:	ISCN Ltd.

From Business Goals to Improvement Planning:
Practical Use of ami* in Industry

INTRODUCTION

The major Software Process Improvement (SPI) conferences and forums have highlighted the lack of improvement planning strategies. When you actually coordinate such SPI initiatives, the real issues situate at the end of the assessment: translation of the assessment results into concrete improvement actions namely improvement action planning. And this exercise is much more tricky than trying to cover straigthly not existing and deficient CMM practices. This is what we will try to address in the present article by introducing the ami* approach (Application of Metrics in Industry), one possible strategy for structuring improvement program, driven by goals and metrics

AMI* IN THE CONTEXT OF STRUCTURING SPI PROGRAM

AMI* IN THE CONTEXT OF STRUCTURING SPI PROGRAM ami* (application of metrics in industry) was a two-year project which started in December 1990 under sponsorship of DG XIII of the Commission of the European Communities through the ESPRIT program promoting the use of measurement in software development. The goal of the project was to develop a practical approach for implementing software measurement and to validate it on a variety of projects all over Europe [AMI95].
The main bottleneck in the majority of SPI programs has been the strategy to adopt after running a capability assessment. The ami* concepts and framework (Assess, Analyze, Metricate, Improve) although defined for implementing software measurement, can be easily applied for structuring improvement program: goal-driven, supported by metrics. As a matter of fact, measurement should play a major role in any SPI initiative:
Provide a quantitative basis of comparison for future changes Help in better understand the issues that may or may not have been identified in the qualitative analysis (assessment) e.g. high cost of certain activities may make the priority improvement targets Make decision making process less risky.

CASE STUDY 1

This case study only covers the assessment and action planning phase. The ami* concepts like goal tree (refer to figure 2) have been applied later in the process to overcome the issues of senior management commitment. The following four main lessons have been learnt during this experience:

Lessons learned 1: The management even exposed to SPI concepts should have been briefed upfront what they have to do concretely during the assessment, before and after. The "ideal" talk from SPI suppliers did not bring them to think in terms of business objectives, critical areas, that are the customers of the SPI project. If the last project failed to deliver the product on time due to a deficient configuration management system, this should drive the forthcoming actions towards the current release. If a platform development starting up now is critical for the survival of the organization, then the emphasis should be first in requirements engineering of this project!! SPI should not be viewed as an independent team. Furthermore, business management should be involved.

Lessons learned 2: Bringing the management to show a rough draft of action plan without having investigated the details of the problem was not appropriate. What could we expect from management? confirmation that we were on the right track?: not with the level of information that was provided. The link to business goals was completely missing, a list of improvement goals per KPA (Key Process Area) were available when problem-oriented goals would have been more appropriate.

Lessons learned 3: Providing an overall schedule & effort estimation for the next year was very much doubtable. In a software project, it is of usual practices to start with a high level plan and effort estimation and then as soon as requirements gets further analyzed, do some detailed planning. The authors are not sure that this approach is also fully applicable for SPI project. At the beginning, requirements are much too vague and no history is available. A better strategy would be first to differentiate across the issues what is from the first sight short, mid or long term and the associated level of effort (low, mid, high, depends on analysis). Then for each major finding, a detailed root-cause analysis should be performed (if necessary further collection of metrics), describe the target process or vision (where do I want to be) and then do a detailed planning for the next 3 to 5 months and some high level milestones for a one-year period.

Lessons learned 4: Metrics were defined at the same time than the goal. Then collection mechanism could be put in place rapidly. The management decided to refine goals with quantitative targets. Due to the variety of projects, those targets were set up for each major product range or markets. This was more appropriate than setting up objectives for the whole organization which would be longer to achieve and consequently less motivating for the project staff.

Bringing back the SPI program on track had to do with linking it to daily business issues: business goals, measurement of performance to evaluate changes, taking critical projects as SPI customers. The measurement program will allow to keep the SPI initiative alive together with continuous management follow-up. A more detailed action plan is now to be worked out and implemented alike any software project.

CASE STUDY 2
The second case study is an interesting one in the sense that it demonstrates that the ami* approach can really solve the problem of deriving efficient actions. It also highlights the role of linking SPI to business goals.
The company is a defense market actor of one of the European countries. The software development staff is about 70 people, less than 10% of the total number of employees. Software is one component of the systems which encompass several other specialities. The company started a process assessment according to the CMM Model in 1994 and was assisted in doing that by one of the most famous US authorized organization. All the participants reported about the professional features of the assessment instruments used and about the skills of the assessors. To follow up with the assessment results, a brainstorming seminar was then organized by the company in order to formalize action plans addressing the main weaknesses observed. This was organized within a 3 months delay which was fitting one of the key success factors of SPI.
But despite this efficient start and some observable improvement by the end of 94 (new procedures in place), the SEPG was not completely satisfied and the senior management was slightly loosing confidence in being able to achieve significant progress without a more structured and tacked approach. At this stage discussions were initiated on how ami* can overcome the problem.

CMM training and assessment based on SEI method (SPA or CBA-IPI) produce consensus among the whole organization on what the major process bottlenecks are, in CMM terms, to achieving the business goals. Agreement from every level and all groups within the organization on what process improvements to undertake systematically is gained. These are the improvements that would benefit quality, productivity, and timeliness most directly as seen by the people who know the current process.
Extensive use of the G/Q/M paradigm as described in the ami* method is cost effective to succeed in defining actions. Steps 4/5/6 of the method (From goals to sub-goals, Verifying the goal tree, From sub-goals to metrics) are applied combined to the assessment findings: Each class of participant (to whom 1 or more sub-goals have been allocated), reviews the assessment findings and answers the following questions:

• Is there a relationship between the sub-goal and the main process weaknesses observed?
• What type of activity could we implement to reach the sub-goal? Is it a short term or mid/long term corrective action?
• What type of action would be effective and efficient in performing sub-goal?
• What type of metrics data would show progress on the action and toward the sub-goal?

As soon as the list of actions is established and validated against the business goals, the complete plan may be prepared. Effort, schedule and owner of each action has to be identified, all figures have then to be compiled before achieving a final check against business goals, overall budget and time frame initially fixed. Priorities may still have to be put in order to fulfill SPI budget constraints. Detailed action plan is completed with the list of metrics to track the goals achievement; for each metric, an exploitation schema is provided i.e. some scenarios to react in case of deviations observed from the initial target.
The above initiative conducted from fall 1995 up till now has largely demonstrated these benefits.
The final goal tree looked as follows: Business Goal: Reduce development Cost

1. Manage the stability of requirements
   1. Reduce the number of change requests
2. Acquire the knowledge of costs
   1. Analyze the reliability of initial estimates
      1. Identify the deviations and origin
   2. Identify how the effort is split along the life cycle phases
      1. Define effort /phase
      2. Identify rework effort
      3. Identify the availability of resources

For each sub goal, actions were defined based on the assessment results. Let's take sub goal "Acquire the knowledge of costs" as an example.
Following the assessment, it was decided to refine the estimation procedure on the basis of a collection of past experiences: this refinement addressed the definition of the structure of the repository for experiences and some mechanisms to reuse this experience. A second action was initiated for the tracking procedure. Therefore it was decided to help software leaders in their analysis of the deviations in order to reduce the risk of not adapted or no corrective actions at all.
The last step of this phase was the definition of indicators. Continuing with the same example, the SEPG came out with the following indicators:
- the deviation rate for an homogeneous group of projects - the deviation rate " for a specific project " observed at each phase of the development (Figure 5). These two indicators were extremely helpful to give visibility in the deviations for projects and to initiate analysis of the reasons for deviation: external occurrence or unrealistic initial estimate or difficulty not forecasted etc.

Metricate

Conclusions of case study 2

This experiment proves that there is a real need in assisting SEPG in deriving action plans from the assessment results. It is not an easy task and without a step by step method there is a big risk of deviation and, at the same time, to loose all the buy in raised during the collaborative work of the assessment. The other main risk is to be too strongly guided by the model and to forget the link with the goals for improvement.
Many organizational units are looking for a certain maturity level by a given time and if you ask to some of the key players the justification for it, he/she will be unable to find a reasonable one. Furthermore, senior managers are usually not motivated by a maturity level but by more economical factors linked to the business. Therefore, a translation is necessary between business goals and process improvements which will hopefully result into measurable products benefits. This translation is straightforward when the ami* approach is used.

CONCLUSION
Those two case studies have shown the necessity to link any software process improvement program to business goals and to follow it up closely with indicators related to the initial goals. Those simple principles have been so far seldom followed by the SPI actors. One reason lies in the lack of emphasis on those issues from SPI suppliers. It might be also related to the inherent characteristics of engineer to tackle any problems as it was a technical problem. SEPGs, Working groups are composed of technical people that have been awarded to low or mid- management position according to their technical results. At level 1-2 of the CMM layer, we are dealing with management-related issues: estimating, planning, controlling, tracking, roles and responsibilities, commitment, ... Maybe that does not excite engineers enough?

References:

[AMI95] Pulford K., Kuntzmann-Combelles A., Shirlaw S.: A Quantitative Approach to Software management, ISBN 0201877465, 1995 , Addison

Table of Contents