简体   繁体   中英

mmap syscall using Golang for reads

I am not an Go unsafe package expert - Neither am I a seasoned C programmer. I am trying to read a huge file > 1G using mmap syscall in go. There are a number of reasons I do mmap and munmap as opposed to read, write I/O. That is beside the point - I can write to the file in a test, when I read from the file, I can ascertain that bytes length matches up, but I cannot read contents of this string file :( Can someone suggest some reading? I need to do to go a little further, here's some code I cooked up for sample test:

filename := "/tmp/dd_file.db"
f, err := os.OpenFile(filename, os.O_RDWR, 0666)
defer f.Close()
if err != nil {
    fmt.Printf("error opening file: %v", err)
}
stat, _ := f.Stat()
size := stat.Size()
fmt.Printf("[READ-ONLY] : size was : %+v\n", size)
got := make([]byte, size)
if _, err := f.ReadAt(got, 0); err != nil && err != io.EOF {
    panic(err)
}
want, err := ioutil.ReadFile(filename)
if err != nil {
    fmt.Printf("[READ-ONLY] : ioutil.ReadFile: %v", err)
}
// going to change the file size now, punch in a few things
t := unsafe.Sizeof("")
if err != nil {
    fmt.Println(err)
    os.Exit(1)
}
_, err = f.Seek(int64(t-1), 0)
if err != nil {
    fmt.Println(err)
    os.Exit(1)
}
_, err = f.Write([]byte(" "))
if err != nil {
    fmt.Println(err)
    os.Exit(1)
}
mmap, err := syscall.Mmap(int(f.Fd()), 0, int(t), syscall.PROT_READ|syscall.PROT_WRITE, syscall.MAP_SHARED)
if err != nil {
    fmt.Println(err)
    os.Exit(1)
}
 // not too sure on reading data on string - doesnt work as expected.
map_array := (*[10000]string)(unsafe.Pointer(&mmap[0]))
map_array[0] = "yellow!"
err = syscall.Munmap(mmap)
if err != nil {
    fmt.Println(err)
    os.Exit(1)
}
newStat, _ := f.Stat()
newSize := newStat.Size()
fmt.Printf("[mmap( ) RW] : size was : %+v\n", newSize)
got = make([]byte, newSize)
if _, err := f.ReadAt(got, 0); err != nil && err != io.EOF {
    panic(err)
}
if len(got) == len(want) {
    fmt.Println("well the lengths are equal atleast??!")
}
if !bytes.Equal(got, want) {
    fmt.Printf("\n [mmap( ) RW] : works! got  %d \n want %d", len(got), len(want))
}

This obviously works as expected - but what if I wanted to read via mmap( ) on an mmapped file, how do I read string out of these bytes (I have a sense there is an encoding package somewhere that I might have to put to use perhaps but then StringHeader on unsafe documentation confused me).

Suggestions.

As @putu noted in a comment, a byte slice can be converted to a string by a simple type conversion:

asStr = string(byteSlice) // entire slice as a string
partStr = string(byteSlice[:100]) // first 100 bytes as a string

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM