An assistive device for the hearing-impaired is proposed to detect and visually display the direction of incidence when a sudden loud sound occurs out of sight. The result is intuitively presented in four angular regions, such as the front, back, left, right regions, through the three procedures; decision of the sound occurrence, reduction of reflections, and direction estimation. Basic concepts of the direction estimation is to use the four directional microphones steering toward corresponding angular regions, and each directional microphone is realized by the array signal processing based on the modified LCMV beamformer. The result is obtained by comparing the beamformer output levels, and leads our attention to corresponding region using LEDs only when a sudden loud sound is detected. The detection is performed using the change of the overall sound pressure level with respect to time as a feature, and its performance and latency are affected by the length of temporal integration. To keep the performance with short latency, an amplitude follower is applied.