- Let PI (for Potential Inputs) denote the set of all sequences of tokens that can be produced by the lexer for a trivial programming language which we will specify below.
lexer
noun; compounding
a program that transforms text into a series of tokens, or units of text that a computer can process easily “lexer” is a short word and is a lot quicker than a phrase like “text-transformer” or “text-processor”
Etymology : lex + er
Source : COMP 311 Lecture notes
Last modified: 10 June 2008