Multimodal biometrics recognition using multi-layer fusion convolutional neural network with RGB and texture descriptor

Abstract

Most existing deep learning architectures rely on feature concatenation or weights combination to construct a representation layer to recognize distinct image biometrics. Using a concatenation operator is inefficient, as it does not effectively cooperate with the multimodal data to achieve adequate recognition performance. Therefore, this motivates the need for a fusion algorithm. This dissertation proposes a novel approach by utilizing a multi-layer fusion algorithm with multimodal biometrics through facial and periocular modalities with texture descriptors. This approach discovers complementary information within the multimodal biometrics data. The multi-layer fusion algorithm is effective in multimodal scenarios, resulting in effective learning. We incorporate multi-layer fusion by correlating multimodal data with its texture descriptors. Through our experiments using public datasets, our proposed approach improved over baseline methods especially when subjected to environmental variations such as aging, illumination, poses, and occlusions. These results also proved that a flexible fusion approach could effectively mitigate the shortcomings of using raw data for identifying individuals, thereby improving on recognition performances across challenging datasets and unconstrained environments.