DNN layer-driven XAI attention mechanism and onboard DL acceleration system for satellite imagery analysis

Abstract

The conventional satellite imagery analysis is mainly operated by manual analysis from human experts. In such operating system, raw data is just sent to the ground station on-demand of analysis by human experts. However, it takes quite a long time for human experts to analyze a whole scene images collected from the satellite. From such problem, the conventional system attempts to analyze only the regions of interest based on the empirical knowledge of human experts. Therefore, it is hard to conduct high-level analysis which requires much more man power and time cost, and is even hard to expand target regions and timelines for complete analysis. Moreover, as it takes several years to train the human experts for analyzing the satellite imagery, the difficulty in taking over such empirical knowledge also remains. In Chapter 1 and 2 of the dissertation, in order to overcome such difficulties, we suggest an explainable AI (XAI) based satellite imagery analysis system that can automate the data refinement at the satellite onboard system and conduct further analysis at ground station's high performance computing (HPC) system (expand the manageable scale), and provide explanations of the analyzed results to the human experts for assisting their final decision making (provide reliability on analysis). In order to realize such system, we introduce the 3 main required technologies in this dissertation: (1) a XAI model for analyzing the satellite imagery with reliability, (2) DL model compression for deploying/accelerating the data refinement at the restricted satellite onboard system, (3) resource management for accelerating further DL inference serving in GPU cluster of the ground station system. In Chapter 3, we propose a method of mediating visual explanations from various attention episodes (i.e. policies of applying attention modules) to improve the explainability in the process of satellite imagery analysis. To generate such various attention episodes in a computing efficient way, we also propose a network layer (DropAttention) that can generate various attention episodes while guaranteeing the stable task performance by training only a single amortized model. From the multiple episodes pool generated by DropAttention, by quantitatively evaluating the explainability of each visual explanation and expanding the parts of explanations with high explainability recursively, our visual explanations mediation scheme attempts to adjust how much to reflect each episodic layer-wise explanation for enforcing a dominant explainability of each candidate. On the empirical evaluation, our methods show their feasibility on enhancing the visual explainability by reducing average drop about 17% and enhancing the rate of increase in confidence 3%. In Chapter 4, we address the problem of deploying/serving the DL-based analysis model that contains higher computational complexity than the capability of resources at the satellite onboard system. As one of the solutions, we propose a new method of controlling the layer-wise channel pruning in a single-shot manner that can decide how much channels to prune in each layer by observing dataset once without full pretraining. To improve the robustness of the performance degradation with regards to the compression rate, we also propose a layer-wise sensitivity and formulate the optimization problems for deciding layer-wise pruning ratio under target computational constraints. We theoretically derive the optimal conditions and propose the practical optimum searching schemes. Through experiments, the proposed methods show robustness on performance degradation, and achieve at most x8 acceleration and x2.6 reduction on memory occupation for processing DL model on the restricted onboard system while preserving the accuracy. In Chapter 5, we also attempt to accelerate XAI-based satellite imagery analysis through satellite - ground station collaborative scheduling. In the subject, we construct a power constraints model for satellite onboard processing, and derived delay model for processing XAI task over satellite and ground station system. Under such models, we formulate optimization problem over satellite and ground station systems, where accelerating DL-based data refinement under power constraints is considered at satellite system, and maximizing the size of episode generation for enhancing the XAI-based analysis performance is considered at ground station system. To solve the optimization problem, we propose a satellite - ground station collaborative scheduling scheme that can accelerate the task of XAI analysis and enlarge the advantage of conducting data refinement at satellite system. Experimental results show that the proposed scheduling scheme achieves x2.1 acceleration and x2.4 lower power consumption than the conventional scheduling scheme at maximum. In conclusion, we developed a XAI model for providing reliability on the process of satellite imagery analysis, and also developed the technologies required to accelerating such analysis models on the satellite on-board system and the GPU cluster system. We presented the superiority of our methods through theoretical analysis and conducting the empirical evaluations.