简体   繁体   English

在没有strtok / lexer的情况下将字符串解析为标记

[英]parsing a string into tokens without strtok/lexer

I want to parse a string into an array of tokens . 我想将字符串解析为令牌数组。 '\\n' and ';' '\\ n'和';' are delimiters , for eg : 是分隔符,例如:

hello;hello
world

should be converted to an array containing: {"hello","hello","world"} . 应该转换为包含以下内容的数组: {"hello","hello","world"}

I tried many different methods for doing this and always I fail (since it needs a dynamic array of char * I have trouble with implementing it). 我尝试了许多不同的方法来执行此操作,但始终失败(因为它需要一个动态的char数组*实现起来很麻烦)。

Please note that I cannot use strtok or lexical analyzer. 请注意,我不能使用strtok或词法分析器。

How may I do this ? 我该怎么办? Any points ? 有什么要点吗?

EDIT : here is one of methods I tried to use but I get segmentation fault (maybe a memory access issue somewhere in my code) : 编辑:这是我尝试使用的方法之一,但是我遇到了段错误(也许是代码中某处的内存访问问题):

#include <stdio.h>
#include <malloc.h>
#include <fcntl.h>
#include <string.h>

typedef struct { 
    int fd;
    char *path;
    int size;
    char *mem;
    struct stat st;
} file;

file *readfile(char *path) {
    file *a=malloc(sizeof(file));
    a->path=path;
    a->fd=open(a->path,O_RDONLY);
    if(a->fd<0) return 0;
    fstat(a->fd,&a->st);
    a->size=a->st.st_size;
    a->mem=malloc(a->size);
    read(a->fd,a->mem,a->size);
    return a;
}

void releasefile(file *a) {
    free(a->mem);
    close(a->fd);
    free(a);
}

char **parse(int *w,file *a) {
    int i,j=0;
    w=0;
    for(i=0;i<=a->size;i++) {
        if(a->mem[i]=='\n' || a->mem[i]==';') { a->mem[i]='\0'; j++; }
    }
    char **out=malloc(sizeof(char *)*j);
    for(i=0;i<=a->size;i++) {
       if(a->mem[i-1]!='\0') continue;
       out[*w]=malloc(strlen(a->mem+i)+1);
       memcpy(out[*w],a->mem+i,strlen(a->mem+i)+1);
       w++;
           return out;
}

int main(int argc,char **argv) {
    file *a=readfile(argv[1]);
    int *w=malloc(sizeof(int));
    char **tokens=parse(w,a);
    int i;
    for(i=0;i<=*w;i++) {
        puts(tokens[i]);
        }
        releasefile(a);

    // ATM no need to check for mem leaks :)

}

Algorithm description : read file, put \\0 where you see a delimiter, start and push tokens seprated by \\0 into an array. 算法描述:读取文件,在\\\\看到一个定界符的地方放置\\ 0,将\\\\分隔的令牌启动并推送到数组中。

What has happened to computer science? 计算机科学发生了什么?

Anyway write a FSA - http://en.wikipedia.org/wiki/Finite-state_machine 无论如何都要写FSA- http://en.wikipedia.org/wiki/Finite-state_machine

Can do this using a table 可以用桌子做

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM