Monotone clustering with sparse generalized additive model

Abstract

Clustering complex data presents significant uncertainty, particularly in cluster interpretation. In many practical scenarios, it is often desired to interpret discovered clusters in an ordered fashion. For example, in healthcare, doctors aim to categorize patients into high-, medium-, and low-risk groups. To address this challenge, we introduce “monotone clustering”, a novel method that identifies inherently ordinal clusters from high-dimensional data. The essence of monotone clustering lies in ensuring that cluster labels are monotonically related to each input variable. We utilize a generalized additive model fortified with monotone splines. Recognizing that not all input variables might influence the ordinal clusters, we incorporate a sign-coherent sparse group penalty on the spline coefficients. This approach aids in highlighting crucial variables and eliminating noise or irrelevant ones. Our algorithm iteratively refines nonlinear monotone functions for the generalized additive model based on existing ordinal clusters and revises cluster assignments using model predictions. The effectiveness and superiority of our monotone clustering approach are substantiated through simulation studies and two real-world examples.