How can I use multiple lexers to scan my input (how do I get “lexer states”)?
Location: http://www.jguru.com/faq/view.jsp?EID=93709 Created: Jul 2, 2000 Modified: 2000-07-02 13:12:22.628 Author: Terence Parr (http://www.jguru.com/guru/viewbio.jsp?EID=1) Question originally posed by Terence Parr PREMIUM (http://www.jguru.com/guru/viewbio.jsp?EID=1 Complicated input languages often have a few lexical constructs that are really nested lexical structures such as Java’s javadoc comments, regular comments, or even just strings. In other words, the allowable characters and structure within the comment or string is radically different from the input surrounding that lexeme. In automaton-based lexers, you use multiple states to handle these situations. Upon “open comment”, you switch to a different state and then have that state switch you back to the normal state upon “close comment”. This is like launching another lexer to handle the nested lexical structure. ANTLR’s approach is to do just that: have a different lexer for each nested lexical substructure. You can even
