【已记录】使用flinksql读取doris时,报内存泄漏

Viewed 60

运行环境

组件 版本号
Doris 2.1.1
Flink 1.8.0
flink-doris-connector-1.18 1.6.0

flinksql

CREATE TABLE doris_test_user (id INT,username STRING,password STRING,create_time TIMESTAMP) WITH ('connector'='doris', 'fenodes'='192.168.1.2:8030', 'username'='root', 'password'='root', 'table.identifier'='aa.ods_users');

 CREATE TABLE doris_test_users1 (id INT,username STRING,password STRING,create_time TIMESTAMP,PRIMARY KEY(id) NOT ENFORCED) WITH ('connector'='jdbc', 'url'='jdbc:mysql://10.1.4.7:3306/test?tinyInt1isBit=false&transformedBitIsBoolean=false', 'driver'='com.mysql.cj.jdbc.Driver', 'username'='root', 'password'='root', 'table-name'='test_users1');

 INSERT INTO doris_test_users1 SELECT id,username,password,create_time FROM doris_test_user;

异常信息

Caused by: java.lang.RuntimeException: SplitFetcher thread 0 received unexpected exception while polling the records

 	at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:168) ~[flink-connector-files-1.18.1.jar:1.18.1]

 	at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:117) ~[flink-connector-files-1.18.1.jar:1.18.1]

 	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[?:?]

 	at java.util.concurrent.FutureTask.run(Unknown Source) ~[?:?]

 	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]

 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]

 	at java.lang.Thread.run(Unknown Source) ~[?:?]

 Caused by: java.lang.IllegalStateException: Memory was leaked by query. Memory leaked: (256)

 Allocator(ROOT) 0/256/256/2147483647 (res/actual/peak/limit)

 

 	at org.apache.doris.shaded.org.apache.arrow.memory.BaseAllocator.close(BaseAllocator.java:477) ~[?:?]

 	at org.apache.doris.shaded.org.apache.arrow.memory.RootAllocator.close(RootAllocator.java:29) ~[?:?]

 	at org.apache.doris.flink.serialization.RowBatch.close(RowBatch.java:496) ~[?:?]

 	at org.apache.doris.flink.serialization.RowBatch.readArrow(RowBatch.java:147) ~[?:?]

 	at org.apache.doris.flink.source.reader.DorisValueReader.hasNext(DorisValueReader.java:246) ~[?:?]

 	at org.apache.doris.flink.source.reader.DorisSourceSplitReader.fetch(DorisSourceSplitReader.java:57) ~[?:?]

 	at org.apache.flink.connector.base.source.reader.fetcher.FetchTask.run(FetchTask.java:58) ~[flink-connector-files-1.18.1.jar:1.18.1]

 	at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.runOnce(SplitFetcher.java:165) ~[flink-connector-files-1.18.1.jar:1.18.1]

 	at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcher.run(SplitFetcher.java:117) ~[flink-connector-files-1.18.1.jar:1.18.1]

 	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[?:?]

 	at java.util.concurrent.FutureTask.run(Unknown Source) ~[?:?]

 	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]

 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]

 	at java.lang.Thread.run(Unknown Source) ~[?:?]
1 Answers

【问题状态】处理中
【问题处理】内部定位中,有进展再更帖