@Prince Sindhiya I have explained it in the ans... Pls correct me if I am wrong
@Prince Sindhiya as the length of the identifier is not fixed that is why we take variable length identifier... Sorry I am not getting what u are trying to ask .. can u pls elaborate ur doubt
to model the structure of tokens with variable length identifier included.
I am asking why lexemes why not tokens?
@Prince Sindhiya at lexical analysis phase we are dealing with lexems after analysis of lexemes we get tokens.A lexeme is a sequence of characters in the source program that matches the pattern for a token and is identified by the lexical analyzer as an instance of that token. A token is a pair consisting of a token name and an optional attribute value.
64.3k questions
77.9k answers
244k comments
80.0k users