Can the parser force a switch to a new lexical state?
Yes, but it is very easy to create bugs by doing so. You can call the token manager’s method SwitchTo from within a semantic action in the parser like this { token_source.SwitchTo(name_of_state) ; } However, owing to look-ahead, the token manager may be well ahead of the parser. Consider Figure 1.2; at any point in the parse, there are a number of tokens on the conveyer belt, waiting to be used by the parser; technically the conveyer belt is a queue of tokens held within the parser object. Any change of state will take effect for the first token not yet in the queue. Syntactic look-ahead usually means there is at least one token in the queue, but there may be many more. If you are going to force a state change from the parser be sure that, at that point in the parsing, the token manager is a known and fixed number of tokens ahead of the parser, and that you know what that number is. If you ever feel tempted to call SwitchTo from the parser, stop and try to think of an alternative metho
