查询 tpch 测试库 lineitem 表,6亿数据报[CANCELLED]FullGC release wg overcommit mem

Viewed 97

Doris 2.1.0,Doris 2.1.1,Doris 2.1.2,Doris 2.1.3,Doris 2.1.4-pre-releas / oracle jdk-1.8.0-201 /linux-7.9 会报错[CANCELLED],内存不足,
但在 Doris 2.0.11 / oracle jdk-1.8.0-201 /linux-7.9 可以正常执行

以下是测试代码:

HikariConfig config = new HikariConfig();
config.setJdbcUrl("jdbc:mysql://192.168.56.121:9030/tpch" + DORIS_CONNECTION_PARAMS);
config.setUsername("root");
config.setPassword("root");
HikariDataSource hikariDataSource = new HikariDataSource(config);
Connection connection = hikariDataSource.getConnection();

PreparedStatement statement = connection.prepareStatement(sql, ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);

statement.setFetchSize(Integer.MIN_VALUE);

ResultSet resultSet = statement.executeQuery();

int i = 1;
while (resultSet.next()) {
    /*String id = resultSet.getString("l_orderkey");
    System.out.println(id);*/
    if (i % 100000 == 0) {
        System.out.println(i);
    }
    i++;
}

System.out.println("流式读取资源结束");

resultSet.close();
statement.close();
connection.close();

报错信息:
Exception in thread "main" java.sql.SQLException: errCode = 2, detailMessage = (192.168.56.121)[CANCELLED]FullGC release wg overcommit mem, wg id:1, name:normal, used:19.96 GB,limit:8.44 GB,backend:192.168.56.121. cancel top memory used tracker <Query#Id=947104c521024a98-a10fa3843706e043> consumption 19.96 GB. details:os physical memory 31.26 GB. process memory used 27.36 GB, limit 28.14 GB, soft limit 25.32 GB. sys available memory 1.63 GB, low water mark 1.60 GB, warning water mark 3.20 GB. Refresh interval memory growth 0 B
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:965)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3978)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3914)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:871)
at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1999)
at com.mysql.jdbc.RowDataDynamic.nextRecord(RowDataDynamic.java:374)
at com.mysql.jdbc.RowDataDynamic.next(RowDataDynamic.java:354)
at com.mysql.jdbc.ResultSetImpl.next(ResultSetImpl.java:6313)
at com.zaxxer.hikari.pool.HikariProxyResultSet.next(HikariProxyResultSet.java)
at com.ssnh.jdbc流式查询.DorisJdbcStreamReader.streamRead(DorisJdbcStreamReader.java:65)
at com.ssnh.jdbc流式查询.DorisJdbc流式查询测试.main(DorisJdbc流式查询测试.java:17)

希望社区可以帮忙下,是需要设置参数么?

2 Answers

资源空闲时,单独跑对应tpcds sql时资源使用率对比如何?

为更具体地了解问题(处理后会更新回帖),可以➕一下W:yz-jayhua