Lexical Analyser Generators
Regular expressions can be converted to non-deterministic finite automata (NFA)
NFAs can be converted in deterministic finite automata (DFA)
DFAs easily implemented as a program
- that apply regular expressions to character sequences, returning tokens
So easy, in fact, that programs can do this automatically from a lexical specification