A pattern-based outlier detection method identifying abnormal attributes in software project data

Abstract

Despite the importance of the quality of software project data, problematic data inevitably occurs during data collection. These data are the outliers with abnormal values on certain attributes, which we call the abnormal attributes of outliers. Manually detecting outliers and their abnormal attributes is laborious and time consuming. Although few existing approaches identify outliers and their abnormal attributes, these approaches are not effective in (1) identifying the abnormal attributes when the outlier has abnormal values on more than the specific number of its attributes or (2) discovering accurate rules to detect outliers and their abnormal attributes. In this paper, we propose a pattern-based outlier detection method that identifies abnormal attributes in software project data: after discovering the reliable frequent patterns that reflect the typical characteristics of the software project data, outliers and their abnormal attributes are detected by matching the software project data with those patterns. Empirical studies were performed on three industrial data sets and 48 artificial data sets with injected outliers. The results demonstrate that our approach outperforms five other approaches by an average of 35.27% and 107.5% in detecting the outliers and abnormal attributes, respectively, on the industrial data sets, and an average of 35.44% and 46.57%, respectively on the artificial data sets. (C) 2009 Elsevier B.V. All rights reserved.