【已记录】doris-streamloader运行out of memory

Viewed 72
2.1
松鼠找不到松果13
2 Answers
张彬华7563

补充一下调试core文件的堆栈信息

 0  0x00000000004674a1 in runtime.raise
    at /usr/local/go/src/runtime/sys_linux_amd64.s:154
 1  0x000000000044a845 in runtime.dieFromSignal
    at /usr/local/go/src/runtime/signal_unix.go:883
 2  0x000000000044ade6 in runtime.sigfwdgo
    at /usr/local/go/src/runtime/signal_unix.go:1096
 3  0x00000000004494e7 in runtime.sigtrampgo
    at /usr/local/go/src/runtime/signal_unix.go:432
 4  0x00000000004674a1 in runtime.raise
    at /usr/local/go/src/runtime/sys_linux_amd64.s:153
 5  0x000000000044a845 in runtime.dieFromSignal
    at /usr/local/go/src/runtime/signal_unix.go:883
 6  0x000000000044aa58 in runtime.crash
    at /usr/local/go/src/runtime/signal_unix.go:975
 7  0x0000000000434737 in runtime.fatalthrow.func1
    at /usr/local/go/src/runtime/panic.go:1131
 8  0x000000000043468c in runtime.fatalthrow
    at /usr/local/go/src/runtime/panic.go:1120
 9  0x000000000043433d in runtime.throw
    at /usr/local/go/src/runtime/panic.go:1047
10  0x00000000004176db in runtime.sysMapOS
    at /usr/local/go/src/runtime/mem_linux.go:187
11  0x00000000004170b5 in runtime.sysMap
    at /usr/local/go/src/runtime/mem.go:142
12  0x0000000000427652 in runtime.(*mheap).grow
    at /usr/local/go/src/runtime/mheap.go:1522
13  0x0000000000426d97 in runtime.(*mheap).allocSpan
    at /usr/local/go/src/runtime/mheap.go:1243
14  0x0000000000426845 in runtime.(*mheap).alloc.func1
    at /usr/local/go/src/runtime/mheap.go:961
15  0x0000000000463ac9 in runtime.systemstack
    at /usr/local/go/src/runtime/asm_amd64.s:496
16  0x0000000000463a60 in runtime.systemstack_switch
    at /usr/local/go/src/runtime/asm_amd64.s:463
17  0x0000000000426785 in runtime.(*mheap).alloc
    at /usr/local/go/src/runtime/mheap.go:955
18  0x0000000000416145 in runtime.(*mcache).allocLarge
    at /usr/local/go/src/runtime/mcache.go:234
19  0x000000000040d03e in runtime.mallocgc
    at /usr/local/go/src/runtime/malloc.go:1053
20  0x000000000044bbe8 in runtime.growslice
    at /usr/local/go/src/runtime/slice.go:266
21  0x00000000006d66fa in doris-gploader/reader.(*FileReader).Read
    at /Users/dxm/work/streamloader/doris-streamloader-1.0.1/reader/reader.go:143
22  0x00000000006ff5be in main.main
    at /Users/dxm/work/streamloader/doris-streamloader-1.0.1/main.go:506
23  0x0000000000436c67 in runtime.main
    at /usr/local/go/src/runtime/proc.go:250
24  0x0000000000465b41 in runtime.goexit
    at /usr/local/go/src/runtime/asm_amd64.s:1598

可以看到是reader.go:143

// copy line into data
data = append(data, line...)

在切片做append的时候导致oom

松鼠找不到松果13