词法分析器无法识别保留字

Question

I'm trying to develop a lexical analyzer which should tokenize and identify operators, identifiers, constants, reserved words, and data types in a code from an external text file, but the problem is I can't get it to identify reserved words or data types , it considers them as identifiers instead. I understand that this is because it matches the very first regex, but I still can't think of another way which will recognize both identifiers/variables as well as reserved words. Any ideas?

import re                                 

tokens = []                               
sample_code = open("book.txt","r").read().split()


for word in sample_code:

   
    if re.match("[a-zA-Z]+", word):
        tokens.append([word,'is an Identifier'])

    
    elif re.match("([1-9][0-9]*)|0", word):
        if word[len(word) - 1] == ';': 
            tokens.append([word[:-1], "is Num Constant"])
            tokens.append([';', 'is a Semi-colon'])
        else: 
            tokens.append([word, "is a Num Constant"])
    
    
    elif word in ['str', 'int', 'bool','float','char']: 
        tokens.append([word, 'is a Datatype'])
    

    elif word in '><!*-/+%=':
        tokens.append([word, "is an Operator"])
    

    elif word in ['if','for','break','elif','else','while','then','call','do',
    'endwhile', 'return','void','static','case','throw','private', 'public']:
        tokens.append([word, "is a Reserved Word"])
    
    
print(tokens, '\n') 

`

Answer 1

Simply move your first regex "Identifier" to the end

import re
import numpy as np
import pandas as pd

tokens = []                               
sample_code = open("book.txt","r").read().split()

for word in sample_code:
    
    if re.match("([1-9][0-9]*)|0", word):
        if word[len(word) - 1] == ';': 
            tokens.append([word[:-1], "is Num Constant"])
            tokens.append([';', 'is a Semi-colon'])
        else: 
            tokens.append([word, "is a Num Constant"])
    
    elif word in ['str', 'int', 'bool','float','char']: 
        tokens.append([word, 'is a Datatype'])
    
    elif word in '><!*-/+%=':
        tokens.append([word, "is an Operator"])
    

    elif word in ['if','for','break','elif','else','while','then','call','do',
    'endwhile', 'return','void','static','case','throw','private', 'public']:
        tokens.append([word, "is a Reserved Word"])

    elif re.match("[a-zA-Z]+", word):
        if word[len(word) - 1] == ';':
            tokens.append([word[:-1], "is an Identifier"])
            tokens.append([';', 'is a Semi-colon'])
        else:
            tokens.append([word, "is an Identifier"])

tokens = np.array(tokens)
tokens = pd.DataFrame(tokens, columns=['Token','Token_Type'])
# tokens.to_excel('tokens.xlsx', index=False)
print(tokens)

The output looks like this: Image