Adaptive maximum entropy regularization for connectionist temporal classification

Abstract

This dissertation focuses on Connectionist Temporal Classification (CTC), a fundamental sequence-to-sequence learning method that leverages dynamic programming for mapping input to output sequences. While CTC has played a pivotal role in sequence learning tasks such as automatic speech recognition (ASR) and optical character recognition (OCR), it is hindered by a persistent challenge—its tendency to generate overly narrow output predictions. To mitigate this challenge EnCTC incorporated an entropy maximization-based regularization term alongside the CTC loss. While EnCTC demonstrated its effectiveness in optical character recognition, it introduced a constant weighting factor for the regularization term during training, which could enforce unnecessary ambiguity even for correct predictions in the later stages of training and affect the overall performance. To address this issue, we present Adaptive Maximum Entropy Regularization (AdaMER), a novel approach that dynamically adjusts the impact of entropy regularization throughout the training process. This adjustment is achieved through the use of a gradient-based learnable parameter that serves as the regularization weighting factor. Our experiments, conducted on the LibriSpeech corpus and various OCR benchmark real-world datasets, provide empirical evidence of the efficacy of AdaMER in addressing the challenges associated with CTC-based sequence learning, ultimately improving model performance.