Adaptive layer-wise Vision GNN scheduling for object change detection

Abstract

Recently, with the rapid development of deep learning technology, it has shown excellent results in the field of computer vision. Detecting changes is an important task in remote sensing of the Earth's surface from satellites or airplanes, or detecting or tracking objects in images. However, object change detection has several problems due to the object scale variance and complexity present in the scene, and different imaging conditions. In recent object change detection tasks, features of input images are mainly extracted through CNN or split the image into sequence features for use in the transformer, which is not flexible in extracting features of irregular and complex objects. Therefore, in this study, we try to improve performance by extracting object features using vision GNN, which is the graph neural network applied to computer vision tasks, and applying it to object change detection. GNN grid size and feature map extracted are different depending on the layer depth of the Vision GNN pyramid structure, and in the deeper layer, it is possible to find correlations between distant node features. However, precise feature extraction is not always guaranteed in deep layers. In addition, it is necessary to ensure the robustness of the model because it becomes difficult to detect changes in objects at short time intervals. In the pyramid structure, the proper depth layer can improve feature representation. So, we propose an Adaptive Layer-wise Vision GNN Scheduling that allows us to measure the degree of change according to the input data and select the proper depth layer of Vision GNNs to extract features based on this. The method of selecting the proper depth layer of the Vision GNN depends on the time interval of the input image, and in a short time interval, the degree of change is dynamically measured according to the input and the vision GNN is controlled. In the proper depth layer scheduled in the GNN grid controller, more precise feature extraction is possible and used for the object change detection task. As a result, in the change detection using the proposed method, the F1-Score improved by 0.45 points (ViG 94.26%, proposed model 94.71%) on the LEVIR dataset, and in the multi-object tracking using the proposed method, the MOTP improved by 19.87 points (ViG 50.89%, proposed model 70.76%) on the VisDrone2019-MOT dataset.