Targeted Model Inversion: Distilling style encoded in predictions

Abstract

Previous model inversion (MI) research has demonstrated the feasibility of reconstructing images representative of specific classes, inadvertently revealing additional feature information. However, there are two remaining challenges for practical black-box MI: (1) minimizing the number of queries to the target model, and (2) reconstructing a high-quality input image tailored to an observed prediction vector. We introduce Targeted Model Inversion (TMI), a practical black-box MI attack. Our approach involves altering the mapping network in StyleGAN, which projects an observed prediction vector into a StyleGAN latent representation. Later, TMI leverages a surrogate model that is also derived from StyleGAN to guide instance-specific MI by optimizing the latent representation. These mapping and surrogate networks work together to conduct high-fidelity MI while significantly decreasing the number of necessary queries. Our experiments demonstrate that TMI outperforms state-of-the-art MI methods, demonstrating a new upper bound on the susceptibility to black-box MI attacks.