Software Process Evaluation: A Machine Learning Approach

Ning Chen, Steven C. H. Hoi, Xiaokui Xiao
2011 IEEE ASE 2011, Lawrence, KS, USA

Abstract—
Software process evaluation is essential to improve software development and the quality of software products in an organization. Conventional approaches based on manual qualitative evaluations (e.g., artifacts inspection) are deficient in the sense that (i) they are time-consuming, (ii) they suffer from the authority constraints, and (iii) they are often subjective. To overcome these limitations, this paper presents a novel semi-automated approach to software process evaluation using machine learning techniques. In particular, we formulate the problem as a sequence classification task, which is solved by applying machine learning algorithms. Based on the framework, we define a new quantitative indicator to objectively evaluate the quality and performance of a software process. To validate the efficacy of our approach, we apply it to evaluate the defect management process performed in four real industrial software projects. Our empirical results show that our approach is effective and promising in providing an objective and quantitative measurement for software process evaluation.

Keywords-software process; defect management process; sequence classification; machine learning

I. INTRODUCTION

problem: how to evaluate a software process performed in a specified scope?

the existing software process evaluation methods often suffer from numerous limitations.

• First, they require manual evaluation, which is rather usually time-consuming, especially for a software development organization with a large number of projects.
• Second, the existing methods suffer from the authority constraints.
• Third, the existing approaches are often based on
  subjective evaluation, and hence, they suffer from biased
  evaluation results due to human factors in the evaluation
  process.

In summary,this paper makes the following contributions:
-• We propose a novel machine learning approach that may
help practitioners to evaluate their software processes.
-• A new quantitative indicator, referred to as process execution qualification rate, is proposed to evaluate the quality and performance of software processes objectively.
-• We compare and explore different kinds of sequence
classification algorithms for solving this problem.

The rest of the paper is organized as follows. Section II
discusses the related work. Section III presents the problem
statement. Section IV introduces the proposed machine learning
approach to software process evaluation tasks. Section V
presents the experimental results and discusses the limitations
of validation. Finally, Section VI concludes this paper.

II. RELATED WORK

III. PROBLEM STATEMENT

IV. APPROACH

A. Overview

Fig. 1 illustrates the proposed framework of
our machine learning approach to software process evaluation.

the overall framework

V. EXPERIMENTS