我正在尝试在 C 中设计一个词法分析器

Question

My Code:

#include <stdio.h>

#include <ctype.h>

#include <string.h>


int main()

{

    int identifier_counter=0,constants_counter=0,operators_counter=0,delimeters_counter=0,i,j,x;
    char expr[50],operators[50],identifiers[50],constants[50],delimeters[50];
    printf("Enter the Expression: ");
    scanf("%[^\n]s",expr);
    for(i=0;i<strlen(expr);i++)
    {
        if(isspace(expr[i])){
            continue;
        }
        else if(isalpha(expr[i]))
        {
            identifiers[identifier_counter]=expr[i];
            identifier_counter++;
        }
        else if(isdigit(expr[i]))
        {
            x=(expr[i]-'0');
            i=i+1;
            while(isdigit(expr[i]))
            {
                x=x*10+(expr[i]-'0');
                i++;
            }
            i=i-1;
            constants[constants_counter]=x;
            constants_counter++;
        }
        else if(expr[i]==','||expr[i]==';'||expr[i]=='{'||expr[i]=='}')
        {
            if(expr[i]==',')
            {
                delimeters[delimeters_counter]=',';
                delimeters_counter++;
            }
            else if(expr[i]==';')
            {
                delimeters[delimeters_counter]=';';
                delimeters_counter++;
            }
            else if(expr[i]=='{')
            {
                delimeters[delimeters_counter]='{';
                delimeters_counter++;
            }
            else if(expr[i]=='}')
            {
                delimeters[delimeters_counter]='}';
                delimeters_counter++;
            }
            
        }
        else
        {
            if(expr[i]=='*')
            {
                operators[operators_counter]='*';
                operators_counter++;
            }
            else if(expr[i]=='-')
            {
                operators[operators_counter]='-';
                operators_counter++;
            }
            else if(expr[i]=='+')
            {
                operators[operators_counter]='+';
                operators_counter++;
            }
            else if(expr[i]=='=')
            {
                operators[operators_counter]='=';
                operators_counter++;
            }
            else if(expr[i]=='/')
            {
                
                operators[operators_counter]='/';
                operators_counter++;
            }
            else if(expr[i]=='%')
            {
                
                operators[operators_counter]='%';
                operators_counter++;
            }
        }
    }
    printf("\nIdentifiers are: ");
    for(j=0;j<identifier_counter;j++)
    {
        printf("%c ",identifiers[j]);
    }
    printf("\nConstants are: ");
    for(j=0;j<constants_counter;j++)
    {
        printf("%d ",constants[j]);
    }
    printf("\nDelimeters are: ");
    for(j=0;j<delimeters_counter;j++)
    {
        printf("%c ",delimeters[j]);
    }
    printf("\nOperators are: ");
    for(j=0;j<operators_counter;j++)
    {
        printf("%c ",operators[j]);
    }
    return 0;
}

My Queries:

1. I want to consider numbers or underscores after the variable's name as "identifiers". for example: user input is "a4". I want the program to print "identifiers are: a4". In my code 'a' is considered as an identifier and '4' as a constant. If the user input is: "a 4" then it's ok if it prints 'a' as identifier and '4' as constant, since there's a space between.

2. I want to generate a token for data types. If the user types 'int' it should return that "datatypes are: int". I know one procedure but it's lengthy and time taking, I've tried to search if there are any built-in functions so that they can return the data type, but I couldn't find any.

Can you help me with these two queries and how I should update my code accordingly. Thank you.

Answer 1

You should split your problem into little pieces. First, how do you recognize a identifier? Write a function, that takes a string and returns the true, if it starts with an identifier. If you have done this, make the function return the length of the identifier. Now do the same thing with numbers and everything you want. Now you have build a tokenizer/lexer. I have an implementation (but a bigger one) here .