Modeling retrosynthesis via multi-decoder transformer with set invariant loss

Abstract

Retrosynthesis, predicting reactants from a given product, is a demanding problem in the field of organic chemistry. Discovering the synthesis path of a new chemical compound with desired properties is key in materials development. As a molecule can be represented as a sequence of characters describing atoms or bonds using Simplified-Molecular-Input Line-Entry-System (SMILES), recent studies started to cast the problem into a seq2seq translation problem from a product to its reactants and leverage the success of neural machine translation models such as Transformer. However, our target in this “translation” task is not a simple sequence as in the regular language translation task but rather a “set” of reactant sequences. Hence, the quality of predictions should not be assessed by the order of reactants generated and naively applying the standard models without this consideration would bring performance degradation. In this paper, we propose a novel $\textbf{Set Invariant Loss}$ function to train the retrosynthesis model, which promotes the model to learn to predict reactants in an order invariant fashion. We also devise new Transformer architecture called $\textbf{Multi-Decoder Transformer}$ and its ensemble techniques suitable for such set invariant loss. We validate our set invariant loss against standard cross-entropy loss on top of recent Transformer based models and we achieve state-of-the-art performance among the template-free based baselines on the standard benchmark dataset (USPTO-50K).