Go: How to deal with Memory leaks while returning a CString?

Question

I have the following function signature which then return a JSON string

func getData(symbol, day, month, year *C.char) *C.char {
  combine, _ := json.Marshal(combineRecords)
  log.Println(string(combine))
  return C.CString(string(combine))
}

The Go code is then being called in Python

import ctypes
from time import sleep
library = ctypes.cdll.LoadLibrary('./deribit.so')
get_data = library.getData

# Make python convert its values to C representation.
# get_data.argtypes = [ctypes.c_char_p, ctypes.c_char_p,ctypes.c_char_p,ctypes.c_char_p]
get_data.restype = ctypes.c_char_p

for i in range(1,100):
    j= get_data("BTC".encode("utf-8"), "5".encode("utf-8"), "JAN".encode("utf-8"), "23".encode("utf-8"))
    # j= get_data(b"BTC", b"3", b"JAN", b"23")
    print('prnting in Python')
    # print(j)
    sleep(1)

It works fine as expected on the Python side but I fear memory leaks when the function will be called in a loop at the Python end.

How do I deal with memory leaks? should I return bytes instead of a CString and deal bytes at Python end to avoid memory leaks? I did find this link to deal with it but somehow I do not know the size of JSON string returned after marshalling

Answer 1

You are right, you have to free it by using C.free

https://pkg.go.dev/cmd/cgo

// Go string to C string
// The C string is allocated in the C heap using malloc.
// It is the caller's responsibility to arrange for it to be
// freed, such as by calling C.free (be sure to include stdlib.h
// if C.free is needed).
func C.CString(string) *C.char

Answer 2

The python should look like:

import ctypes
from time import sleep
library = ctypes.CDLL('./stackoverflow.so')
get_data = library.GetData
free_me = library.FreeMe
free_me.argtypes = [ctypes.POINTER(ctypes.c_char)]
get_data.restype = ctypes.POINTER(ctypes.c_char)

for i in range(1,100):
  j = get_data("", "", "")
  print(ctypes.c_char_p.from_buffer(j).value)
  free_me(j)
  sleep(1)

The go should look like:

package main
/*
#include <stdlib.h>
*/
import "C"
import (
  "log"
  "unsafe"
)

//export GetData
func GetData(symbol, day, month, year *C.char) *C.char {
  combine := "combine"
  log.Println(string(combine))
  return C.CString(string(combine))
}

//export FreeMe
func FreeMe(data *C.char) {
  C.free(unsafe.Pointer(data))
}

func main() {}

And use this command line to generate the shared library:

python3 --version
Python 3.8.10 
go version
go version go1.19.2 linux/amd64
go build -o stackoverflow.so -buildmode=c-shared github.com/sjeandeaux/stackoverflow
python3 stackoverflow.py 
2023/01/03 13:54:14 combine
b'combine'                                                                                                                                  
...

FROM ubuntu:18.04

RUN apt-get update -y && apt-get install python -y

COPY stackoverflow.so stackoverflow.so
COPY stackoverflow.py stackoverflow.py

CMD ["python", "stackoverflow.py"]

docker build --tag stackoverflow .
docker run -ti stackoverflow
2023/01/03 15:04:24 combine
b'combine'
...