Differential representation of face pareidolia in human and deep neural network

Abstract

Humans can perceive not only a real face but also an illusory face, a process called face pareidolia. The previous works have suggested that face pareidolia is occurred in primates because of face-detection mechanism. However, previous studies have produced results based on what is happening in the primate brain, making it a little difficult to know whether there was a brain feedback loop or cognitive effect, rather than just a face detection mechanism. To bridge this gap, the study used modern convolutional neural networks (CNNs), a computer model known as deep neural networks, one of the computer models similar to the human visual system. In this paper, we first observed facial pareidolia in the CNNs. To explore the differences in the internal processing representations of visual stimuli that contains face features in the brain and neural network, we used pattern similarity analysis using facelike-, object-, face images. We found that facial pareidolia is a phenomenon that is caused by amount of face visual perception. We also found a high pattern similarity between the back layer of the convolutional neural network and the fusiform face area (FFA) in face pareidolia task. The reason for this is that there are channels that selectively reacts to faces in the back layer of the convolutional network. Experiments with these channels have shown that the greater the impact, the higher the correlation with humans, but no linear results. This suggests that facial pareidolia in the brain is not only due to a face detection mechanism, but also due to the high impact of the FFA region and also the inversely affecting pathways that do not exist in the CNN.