Tokenization in lexical analysis

Question

Consider the following C program:
 

int main (void)
{
    in/*this is an example*/z;
    
    double/*is it an error?*/y;
    
    print( “This is simple” );
    
    return 0;
}

- How many Different tokens are there in the above code.

Comments

please categorize the tokens.

Answer 1

int main ( void )
{
    in z ;
    
    double y ;
    
    print( “This is simple” );
    
    return 0 ;
}

All Colored ones are different Tokens = 16

Types of tokens-  

keywords - int, void , double, return.

Identifiers - main , in, z, y, print.

Strings     - “This is simple”.

Constant  - 0.

Special- symbols / Delimiter / Punctuator /  - ( , ), ; , { , } , comma.

NOTE:

 ( 6 types of  c tokens )

Comments

i have run this on a compiler it takes in and z as seperate tokens ..they will be identifiers

---

I am getting 17 as the answer by counting in the best answer given above.

Can someone help where I am going wrong??

I counted beginning from keyword and my counts are

1. -> 4

2 -> 5

3 -> 1

4 -> 1

5 -> 6

---

the answer is right you added up wrong it's 17 you counted only 16

Answer 2

"int","main","(","void",")"-5 tokens

"{"-1 token

"inz",";"-2 tokens

"double","y",";"-3 tokens

"printf","(","String literal",")",";"-5 tokens

"return","0",";"- 3 tokens

"}"- 1 token

Total 20 tokens

int,void-keyword

"inz","y"-identifier

";"-delimeter

Comments

no token need not be seperated by white space like () = 2 token but no space , white space need when we define by own i.e. not known to computer

---

Anu.. Yes I am getting your point but the example you are giving is two different separators which is considered to be two tokens definitely.. But what about above example.. Plz Cross check it would be very  helpful.. If any reference From book or anywhere..

---

21 token but distinct are 16 as selected answer explain.