Onset-sensitive alignment of piano music using neural AMTs

Abstract

Audio-to-Score alignment refers to matching the notes or symbols in a given performance with the score in time. Precise alignment is essential for a system that extracts or interacts with performance information, and is therefore actively studied. In this paper, we discuss how to employ automatic music transcription result using artificial neural network as a feature for alignment. Even though the results of automatic music transcription are not perfect, the results can be thought of as learned features that can be directly compared to score. In the past, there have been studies using automatic music transcription in alignment. However, only the simplest form was used, and quantitative comparison with the method of using other features or comparison with various transcription method were not made.

In this paper, we compare the performance of automatic music transcription network using three network structures: convolutional neural network, recurrent neural network, and recurrent neural network over convolution layers. Furthermore, we shows the addition of the neural network which predicts only the onset can significantly improve the temporal accuracy of alignment.

For comparison with other methods, we used only dynamic time warping without additional steps, and compared the results with the existing methodology using published data. Experimental results show that the result of the alignment depends on the transcription method and that the accuracy of the transcription and the accuracy of the alignment are not simply proportional. In particular, the best alignment results using the onset showed significantly better performance than those without onset and heuristic features.