Scalable framework for parsing: from Fortress to JavaScript

Abstract

Programming languages grow over time that requires frequent changes in language manipulations such as compilation, interpretation, and analysis. Because the very first step of most language manipulations is parsing, whether parsing can adapt to changes easily, quickly, and correctly, it affects the scalability of language manipulations. Even though various parsing techniques have been well studied theoretically, their practical experiences in scalable frameworks have not been available. In this paper, we present our experiences with parsing in scalable frameworks. We first describe our trials and errors using various parsing techniques in developing parsers for the Fortress programming language. Because Fortress was a scientific language under development, its mathematical and growable syntax introduced new challenges in parsing. We summarize the lessons learned from parsing Fortress, and we share our experience of applying the lessons to parsing the JavaScript programming language. While JavaScript is one of the most widely used languages, JavaScript itself and its diverse variants keep extending its syntax, and the extremely dynamic features of JavaScript also add new challenges in parsing. Using automatic generation tools and methods like staged parsing and automatic extraction and testing of examples in language specifications, our methodology for scalable parsing has shown to be very effective in practice. Copyright (c) 2015 John Wiley & Sons, Ltd