Flex yy_scan_string memory leaks

Question

Description

This is an example created to introduce issue from larger solution. I have to use flex and yy_scan_string() . I have an issue with memory leaks in flex (code below). In this example memory leaks are marked as "still reachable", but in the original solution they get marked as "lost memory".

I think this problem is somewhere in memory allocated by flex internally, which I don't know how to free properly and I can't find any tutorial / documentation for that issue.

file.lex

%option noyywrap                   

%{                    
#include <stdio.h>
%}

%%
. printf("%s\n", yytext);
%%

int main() {
    printf("Start\n");
    yy_scan_string("ABC");
    yylex();
    printf("Stop\n");
    return 0;
}

bash$ flex file.lex
bash$ gcc lex.yy.c
bash$ ./a.out
Start
H
a
l
l
o

W
o
r
l
d
Stop
bash$ valgrind --leak-check=full --show-leak-kinds=all ./a.out
==6351== Memcheck, a memory error detector
==6351== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==6351== Using Valgrind-3.14.0 and LibVEX; rerun with -h for copyright info
==6351== Command: ./a.out
==6351==
==6351== error calling PR_SET_PTRACER, vgdb might block
Start
A
B
C
Stop
==6351==
==6351== HEAP SUMMARY:
==6351==     in use at exit: 77 bytes in 3 blocks
==6351==   total heap usage: 4 allocs, 1 frees, 589 bytes allocated
==6351==
==6351== 5 bytes in 1 blocks are still reachable in loss record 1 of 3
==6351==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==6351==    by 0x10AE84: yyalloc (in ./a.out)
==6351==    by 0x10ABE5: yy_scan_bytes (in ./a.out)
==6351==    by 0x10ABBC: yy_scan_string (in ./a.out)
==6351==    by 0x10AEE2: main (in /home/./a.out)
==6351==
==6351== 8 bytes in 1 blocks are still reachable in loss record 2 of 3
==6351==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==6351==    by 0x10AE84: yyalloc (in ./a.out)
==6351==    by 0x10A991: yyensure_buffer_stack (in ./a.out)
==6351==    by 0x10A38D: yy_switch_to_buffer (in ./a.out)
==6351==    by 0x10AB8E: yy_scan_buffer (in ./a.out)
==6351==    by 0x10AC68: yy_scan_bytes (in ./a.out)
==6351==    by 0x10ABBC: yy_scan_string (in ./a.out)
==6351==    by 0x10AEE2: main (in ./a.out)
==6351==
==6351== 64 bytes in 1 blocks are still reachable in loss record 3 of 3
==6351==    at 0x483577F: malloc (vg_replace_malloc.c:299)
==6351==    by 0x10AE84: yyalloc (in ./a.out)
==6351==    by 0x10AAEF: yy_scan_buffer (in ./a.out)
==6351==    by 0x10AC68: yy_scan_bytes (in ./a.out)
==6351==    by 0x10ABBC: yy_scan_string (in ./a.out)
==6351==    by 0x10AEE2: main (in ./a.out)
==6351==
==6351== LEAK SUMMARY:
==6351==    definitely lost: 0 bytes in 0 blocks
==6351==    indirectly lost: 0 bytes in 0 blocks
==6351==      possibly lost: 0 bytes in 0 blocks
==6351==    still reachable: 77 bytes in 3 blocks
==6351==         suppressed: 0 bytes in 0 blocks
==6351==
==6351== For counts of detected and suppressed errors, rerun with: -v
==6351== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

Answer 1

The answer is mentioned in the flex documentation, but you have to read carefully to find it. (See below.)

If you generate a reentrant scanner, then you are responsible for creating and destroying each scanner object you require, and the scanner object manages all of the memory required by a scanner instance. But even if you don't use the reentrant interface, you can use yylex_destroy to manage memory. In the traditional non-reentrant interface, then there is no scanner_t argument, so the prototype of yylex_destroy is simply

int yylex_destroy(void);

(Although it has a return value which is supposed to be a status code, it never returns an error.)

You can, if you want to, call yylex_init , which also takes no arguments in the non-reentrant interface, but unlike the reentrant interface, it is not necessary to call it.

From the manual chapter on memory management :

Flex allocates dynamic memory during initialization, and once in a while from within a call to yylex(). Initialization takes place during the first call to yylex(). Thereafter, flex may reallocate more memory if it needs to enlarge a buffer. As of version 2.5.9 Flex will clean up all memory when you call yylex_destroy See faq-memory-leak .

---

Example:

$ cat clean.l
%option noinput nounput noyywrap nodefault
%{                    
#include <stdio.h>
%}

%%
.|\n ECHO;
%%

int main() {
    printf("Start\n");
    yy_scan_string("ABC");
    yylex();
    printf("Stop\n");
    yylex_destroy();
    return 0;
}

$ flex -o clean.c clean.l
$ gcc -Wall -o clean clean.c
$ valgrind --leak-check=full --show-leak-kinds=all ./clean
==16187== Memcheck, a memory error detector
==16187== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==16187== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==16187== Command: ./clean
==16187== 
Start
ABCStop
==16187== 
==16187== HEAP SUMMARY:
==16187==     in use at exit: 0 bytes in 0 blocks
==16187==   total heap usage: 4 allocs, 4 frees, 1,101 bytes allocated
==16187== 
==16187== All heap blocks were freed -- no leaks are possible
==16187== 
==16187== For counts of detected and suppressed errors, rerun with: -v
==16187== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)