Online speaker segmentation and clustering of spoken documents

Abstract

As a variety of multimedia data such as broadcast news, entertainments, and education materials, are produced every day and spread over the internet, content retrieval technologies have become essential to search and manage such a large amount of data. In relation to this, people are beginning to take interest in spoken document retrieval, as research on speech and speaker recognition has led to major technical breakthroughs with smart devices. Spoken documents contain speech from various speakers and thus speaker diarization or speaker indexing is important for retrieval. Speaker diarization determines how many speakers are included in a given spoken document and partitions the document into homogeneous segments according to each speaker`s identity. This task replies to the question "Who spoke when?", whereas speaker recognition addresses the question "Who spoke?". Speaker diarization consists of three processes, speech detection, speaker segmentation, and clustering segments. This dissertation proposes online speaker segmentation and clustering technique of spoken documents for speaker diarization system. Speaker segmentation is to find the change point of the speakers so that each segment contains only one speaker`s speech. It has various applications such as a preprocessing task for audio indexing, speaker tracking, information extraction, and so on. The most popular criterion used in unsupervised speaker segmentation is the Bayesian Information Criterion (BIC). Conventional BIC-based speaker segmentation firstly constructs two single Gaussian models for two divided speech streams respectively, in an analysis window, a regular size of speech data shifted over the audio stream. And then, the dissimilarity between the two independent models is estimated according to the BIC principle. This approach has been successfully applied to speaker segmentation. However, it tends to fail to detect speaker changes for short speech segments since it is hard to represent...