2.1.8 版本 arm结构飞腾芯片 只运行doris,7-8小时左右,CPU飙升导致机器卡死

Viewed 48

机器配置

CPU:是飞腾5000
操作系统: 麒麟v10 sp3
k8s版本:1.26.16
刀片机, 分配的空间是KVM虚拟化出来的
24核48G

测试路线

docker compose部署、k8s部署、裸机部署

be部分配置

JAVA_OPTS="-Dfile.encoding=UTF-8 -Xmx6g -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintGCCause -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=$LOG_DIR/heapdump.hprof -DlogPath=$LOG_DIR/jni.log -Xloggc:$DORIS_HOME/log/be.gc.log.$CUR_DATE -Djavax.security.auth.useSubjectCredsOnly=false -Dsun.security.krb5.debug=true -Dsun.java.command=DorisBE -XX:-CriticalJNINatives"

priority_networks = 172.20.80.0/24
string_type_length_soft_limit_bytes= 2147483643
doris_scanner_thread_pool_thread_num = 4
doris_scanner_thread_pool_queue_size = 8
max_scan_key_num = 64
pipeline_exec_thread_pool_thread_num = 4
enable_fqdn_mode = true
heartbeat_service_password = Deta2025!
fragment_pool_thread_num_max = 2048
fragment_pool_queue_size = 4096
brpc_num_threads = 256

fe 部分配置

JAVA_OPTS="-Dfile.encoding=UTF-8 -Djavax.security.auth.useSubjectCredsOnly=false -Xss4m -Xmx6g -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -Xloggc:$LOG_DIR/fe.gc.log.$CUR_DATE -Dlog4j2.formatMsgNoLookups=true"

qe_query_timeout_second = 300
qe_slow_log_ms = 5000
priority_networks = 172.20.80.0/24
string_type_hard_limit = 2147483643
table_name_length_limit = 256
metadata_checkpoint_interval_seconds = 3600  # 减少元数据检查点频率
enable_fqdn_mode = true
remote_fragment_exec_timeout_ms = 30000

fe.gc.log 崩溃前的几条

2025-03-24T19:03:08.106+0800: 9450.121: [GC pause (G1 Evacuation Pause) (young)GC Start: NUMA region verification (id: matched/mismatched/total): 0: 0/0/192  
GC End: NUMA region verification (id: matched/mismatched/total): 0: 0/0/192  
, 0.0260024 secs]
   [Parallel Time: 20.6 ms, GC Workers: 4]
      [GC Worker Start (ms): Min: 9450121.9, Avg: 9450122.0, Max: 9450122.1, Diff: 0.2]
      [Ext Root Scanning (ms): Min: 5.8, Avg: 6.1, Max: 6.4, Diff: 0.6, Sum: 24.2]
      [Update RS (ms): Min: 9.6, Avg: 9.6, Max: 9.7, Diff: 0.1, Sum: 38.4]
         [Processed Buffers: Min: 30, Avg: 57.0, Max: 116, Diff: 86, Sum: 228]
      [Scan RS (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.2]
      [Code Root Scanning (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
      [Object Copy (ms): Min: 3.9, Avg: 4.2, Max: 4.4, Diff: 0.5, Sum: 16.9]
      [Termination (ms): Min: 0.2, Avg: 0.3, Max: 0.4, Diff: 0.2, Sum: 1.3]
         [Termination Attempts: Min: 1, Avg: 2.5, Max: 5, Diff: 4, Sum: 10]
      [GC Worker Other (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.1]
      [GC Worker Total (ms): Min: 20.2, Avg: 20.3, Max: 20.4, Diff: 0.1, Sum: 81.1]
      [GC Worker End (ms): Min: 9450142.1, Avg: 9450142.3, Max: 9450142.4, Diff: 0.2]
   [Code Root Fixup: 0.1 ms]
   [Code Root Purge: 0.0 ms]
   [Clear CT: 0.7 ms]
   [Other: 4.6 ms]
      [Choose CSet: 0.0 ms]
      [Ref Proc: 2.6 ms]
      [Ref Enq: 0.0 ms]
      [Redirty Cards: 0.3 ms]
      [Humongous Register: 0.1 ms]
      [Humongous Reclaim: 0.0 ms]
      [Free CSet: 0.1 ms]
   [Eden: 86016.0K(86016.0K)->0.0B(81920.0K) Survivors: 4096.0K->6144.0K Heap: 269.5M(384.0M)->186.8M(384.0M)]
 [Times: user=0.09 sys=0.00, real=0.02 secs] 
2025-03-24T19:03:22.776+0800: 9464.791: [GC pause (G1 Evacuation Pause) (young) (initial-mark)GC Start: NUMA region verification (id: matched/mismatched/total): 0: 0/0/192  
GC End: NUMA region verification (id: matched/mismatched/total): 0: 0/0/192  
, 0.0239032 secs]
   [Parallel Time: 18.4 ms, GC Workers: 4]
      [GC Worker Start (ms): Min: 9464793.1, Avg: 9464793.2, Max: 9464793.4, Diff: 0.3]
      [Ext Root Scanning (ms): Min: 8.0, Avg: 8.5, Max: 9.1, Diff: 1.1, Sum: 34.2]
      [Update RS (ms): Min: 3.9, Avg: 4.1, Max: 4.4, Diff: 0.5, Sum: 16.4]
         [Processed Buffers: Min: 40, Avg: 56.2, Max: 86, Diff: 46, Sum: 225]
      [Scan RS (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.2]
      [Code Root Scanning (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
      [Object Copy (ms): Min: 3.5, Avg: 3.9, Max: 4.3, Diff: 0.8, Sum: 15.4]
      [Termination (ms): Min: 0.1, Avg: 0.2, Max: 0.2, Diff: 0.1, Sum: 0.7]
         [Termination Attempts: Min: 1, Avg: 1.0, Max: 1, Diff: 0, Sum: 4]
      [GC Worker Other (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.1]
      [GC Worker Total (ms): Min: 16.6, Avg: 16.7, Max: 16.8, Diff: 0.2, Sum: 66.9]
      [GC Worker End (ms): Min: 9464809.9, Avg: 9464810.0, Max: 9464810.0, Diff: 0.1]
   [Code Root Fixup: 0.1 ms]
   [Code Root Purge: 0.0 ms]
   [Clear CT: 0.4 ms]
   [Other: 5.0 ms]
      [Choose CSet: 0.0 ms]
      [Ref Proc: 2.8 ms]
      [Ref Enq: 0.0 ms]
      [Redirty Cards: 0.4 ms]
      [Humongous Register: 0.1 ms]
      [Humongous Reclaim: 0.0 ms]
      [Free CSet: 0.1 ms]
   [Eden: 81920.0K(81920.0K)->0.0B(81920.0K) Survivors: 6144.0K->6144.0K Heap: 266.8M(384.0M)->186.1M(384.0M)]
 [Times: user=0.08 sys=0.00, real=0.02 secs] 
2025-03-24T19:03:22.800+0800: 9464.816: [GC concurrent-root-region-scan-start]
2025-03-24T19:03:22.811+0800: 9464.826: [GC concurrent-root-region-scan-end, 0.0108680 secs]
2025-03-24T19:03:22.811+0800: 9464.826: [GC concurrent-mark-start]
2025-03-24T19:03:23.038+0800: 9465.054: [GC concurrent-mark-end, 0.2270705 secs]
2025-03-24T19:03:23.040+0800: 9465.055: [GC remark 2025-03-24T19:03:23.040+0800: 9465.055: [Finalize Marking, 0.0014099 secs] 2025-03-24T19:03:23.041+0800: 9465.057: [GC ref-proc, 0.0011486 secs] 2025-03-24T19:03:23.042+0800: 9465.058: [Unloading, 0.0681576 secs], 0.0719300 secs]
 [Times: user=0.27 sys=0.00, real=0.07 secs] 
2025-03-24T19:03:23.113+0800: 9465.129: [GC cleanup 187M->187M(384M), 0.0035347 secs]
 [Times: user=0.01 sys=0.00, real=0.00 secs] 
2025-03-24T19:03:35.503+0800: 9477.518: [GC pause (G1 Evacuation Pause) (young)GC Start: NUMA region verification (id: matched/mismatched/total): 0: 0/0/192  
GC End: NUMA region verification (id: matched/mismatched/total): 0: 0/0/192  
, 0.0203222 secs]
   [Parallel Time: 15.2 ms, GC Workers: 4]
      [GC Worker Start (ms): Min: 9477519.1, Avg: 9477519.3, Max: 9477519.4, Diff: 0.3]
      [Ext Root Scanning (ms): Min: 6.3, Avg: 6.5, Max: 6.9, Diff: 0.5, Sum: 26.2]
      [Update RS (ms): Min: 4.0, Avg: 4.1, Max: 4.3, Diff: 0.2, Sum: 16.5]
         [Processed Buffers: Min: 27, Avg: 55.5, Max: 79, Diff: 52, Sum: 222]
      [Scan RS (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.1]
      [Code Root Scanning (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.0]
      [Object Copy (ms): Min: 3.6, Avg: 3.9, Max: 4.2, Diff: 0.5, Sum: 15.7]
      [Termination (ms): Min: 0.0, Avg: 0.2, Max: 0.2, Diff: 0.2, Sum: 0.6]
         [Termination Attempts: Min: 1, Avg: 1.0, Max: 1, Diff: 0, Sum: 4]
      [GC Worker Other (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.1]
      [GC Worker Total (ms): Min: 14.5, Avg: 14.8, Max: 15.0, Diff: 0.4, Sum: 59.2]
      [GC Worker End (ms): Min: 9477533.9, Avg: 9477534.1, Max: 9477534.1, Diff: 0.2]
   [Code Root Fixup: 0.1 ms]
   [Code Root Purge: 0.0 ms]
   [Clear CT: 0.5 ms]
   [Other: 4.5 ms]
      [Choose CSet: 0.0 ms]
      [Ref Proc: 2.4 ms]
      [Ref Enq: 0.0 ms]
      [Redirty Cards: 0.4 ms]
      [Humongous Register: 0.1 ms]
      [Humongous Reclaim: 0.0 ms]
      [Free CSet: 0.1 ms]
   [Eden: 81920.0K(81920.0K)->0.0B(12288.0K) Survivors: 6144.0K->6144.0K Heap: 266.1M(384.0M)->187.0M(384.0M)]
 [Times: user=0.06 sys=0.01, real=0.03 secs] 
2025-03-24T19:03:37.792+0800: 9479.807: [GC pause (G1 Evacuation Pause) (mixed)GC Start: NUMA region verification (id: matched/mismatched/total): 0: 0/0/192  
GC End: NUMA region verification (id: matched/mismatched/total): 0: 0/0/192  
, 0.0305925 secs]
   [Parallel Time: 27.8 ms, GC Workers: 4]
      [GC Worker Start (ms): Min: 9479807.9, Avg: 9479811.6, Max: 9479818.8, Diff: 10.8]
      [Ext Root Scanning (ms): Min: 0.1, Avg: 4.6, Max: 9.3, Diff: 9.2, Sum: 18.6]
      [Update RS (ms): Min: 4.2, Avg: 6.9, Max: 9.8, Diff: 5.6, Sum: 27.6]
         [Processed Buffers: Min: 1, Avg: 34.2, Max: 95, Diff: 94, Sum: 137]
      [Scan RS (ms): Min: 0.0, Avg: 1.4, Max: 2.8, Diff: 2.8, Sum: 5.4]
      [Code Root Scanning (ms): Min: 0.0, Avg: 0.0, Max: 0.2, Diff: 0.2, Sum: 0.2]
      [Object Copy (ms): Min: 7.9, Avg: 10.4, Max: 16.6, Diff: 8.7, Sum: 41.7]
      [Termination (ms): Min: 0.0, Avg: 0.2, Max: 0.3, Diff: 0.2, Sum: 0.8]
         [Termination Attempts: Min: 1, Avg: 5.5, Max: 13, Diff: 12, Sum: 22]
      [GC Worker Other (ms): Min: 0.0, Avg: 0.0, Max: 0.0, Diff: 0.0, Sum: 0.1]
      [GC Worker Total (ms): Min: 16.5, Avg: 23.6, Max: 27.2, Diff: 10.7, Sum: 94.3]
      [GC Worker End (ms): Min: 9479835.0, Avg: 9479835.2, Max: 9479835.3, Diff: 0.2]
   [Code Root Fixup: 0.1 ms]
   [Code Root Purge: 0.0 ms]
   [Clear CT: 0.3 ms]
   [Other: 2.4 ms]
      [Choose CSet: 0.0 ms]
      [Ref Proc: 0.8 ms]
      [Ref Enq: 0.0 ms]
      [Redirty Cards: 0.3 ms]
      [Humongous Register: 0.1 ms]
      [Humongous Reclaim: 0.0 ms]
      [Free CSet: 0.1 ms]
   [Eden: 12288.0K(12288.0K)->0.0B(83968.0K) Survivors: 6144.0K->4096.0K Heap: 199.0M(384.0M)->186.4M(384.0M)]
 [Times: user=0.11 sys=0.00, real=0.03 secs] 

fe.log 崩溃前日志

RuntimeLogger 2025-03-24 11:03:34,539 INFO (report-thread|204) [ReportHandler.storagePolicyReport():364] backend[10004] reports policies [], report resources: []
RuntimeLogger 2025-03-24 11:03:34,539 INFO (report-thread|204) [ReportHandler.tabletReport():473] backend[10004] reports 1078 tablet(s). report version: 17428048011056
RuntimeLogger 2025-03-24 11:03:34,543 INFO (report-thread|204) [TabletInvertedIndex.tabletReport():377] finished to do tablet diff with backend[10004]. fe tablet num: 1078, backend tablet num: 1078. sync: 0. metaDel: 0. foundInMeta: 1078. migration: 0. backend partition num: 138, backend need update: 0. found invalid transactions 0. found republish transactions 0. tabletToUpdate: 0. need recovery: 0. cost: 3 ms
RuntimeLogger 2025-03-24 11:03:34,544 INFO (report-thread|204) [ReportHandler.tabletReport():579] finished to handle tablet report from backend[10004] cost: 4 ms
RuntimeLogger 2025-03-24 11:03:35,189 INFO (mysql-nio-pool-42|6362) [DatabaseTransactionMgr.beginTransaction():397] begin transaction: txn id 5724 with label label_2c080caf37de44e3_85eee66ee2c74722 from coordinator FE: 172.20.80.22, listener id: -1
RuntimeLogger 2025-03-24 11:03:35,189 INFO (mysql-nio-pool-42|6362) [OlapInsertExecutor.beforeExec():197] start insert [label_2c080caf37de44e3_85eee66ee2c74722] with query id 2c080caf37de44e3-85eee66ee2c74722 and txn id 5724
RuntimeLogger 2025-03-24 11:03:35,190 INFO (mysql-nio-pool-42|6362) [Coordinator.execInternal():776] dispatch load job: 2c080caf37de44e3-85eee66ee2c74722 to [TNetworkAddress(hostname:172.20.80.5, port:9060)]
RuntimeLogger 2025-03-24 11:03:35,190 INFO (mysql-nio-pool-42|6362) [Coordinator.sendPipelineCtx():1038] num local sink for backend 10004 is 1
RuntimeLogger 2025-03-24 11:03:35,250 INFO (mysql-nio-pool-42|6362) [DatabaseTransactionMgr.commitTransaction():828] transaction:[TransactionState. transaction id: 5724, label: label_2c080caf37de44e3_85eee66ee2c74722, db id: 10162, table id list: 11468, callback id: -1, coordinator: FE: 172.20.80.22, transaction status: COMMITTED, error replicas num: 0, replica ids: , prepare time: 1742814215188, commit time: 1742814215245, finish time: -1, reason: ] successfully committed
RuntimeLogger 2025-03-24 11:03:35,254 INFO (PUBLISH_VERSION|20) [PublishVersionDaemon.genPublishTask():178] send publish tasks for transaction: 5724, db: 10162
RuntimeLogger 2025-03-24 11:03:35,268 INFO (PUBLISH_VERSION|20) [OlapTable.updateVisibleVersionAndTime():2984] updateVisibleVersionAndTime, tableName: audit_operation_logs, visibleVersion, 3713, visibleVersionTime: 1742814215265
RuntimeLogger 2025-03-24 11:03:35,273 INFO (PUBLISH_VERSION|20) [MTMVService.processEvent():185] processEvent, Event: TableEvent{ctlId=0, ctlName='internal', dbId=10162, dbName='doris_db', tableId=11468, tableName='audit_operation_logs'} Event{eventId=33530, eventTime=1742814215273, eventType=DATA_CHANGE}
RuntimeLogger 2025-03-24 11:03:35,273 INFO (PUBLISH_VERSION|20) [DatabaseTransactionMgr.finishTransaction():1110] finish transaction TransactionState. transaction id: 5724, label: label_2c080caf37de44e3_85eee66ee2c74722, db id: 10162, table id list: 11468, callback id: -1, coordinator: FE: 172.20.80.22, transaction status: VISIBLE, error replicas num: 0, replica ids: , prepare time: 1742814215188, commit time: 1742814215245, finish time: 1742814215265, reason:  successfully, publish times 1, publish result QUORUM_SUCC
RuntimeLogger 2025-03-24 11:03:35,299 INFO (mysql-nio-pool-42|6362) [Coordinator.execInternal():756] dispatch result sink of query 4ed91d6c734440a4-b4c500e6e9d02717 to TNetworkAddress(hostname:172.20.80.6, port:9060)
RuntimeLogger 2025-03-24 11:03:35,313 INFO (mysql-nio-pool-44|6692) [DatabaseTransactionMgr.beginTransaction():397] begin transaction: txn id 5725 with label label_51fc822fa1374cad_8d070e80ed8a54f8 from coordinator FE: 172.20.80.22, listener id: -1
RuntimeLogger 2025-03-24 11:03:35,314 INFO (mysql-nio-pool-44|6692) [OlapInsertExecutor.beforeExec():197] start insert [label_51fc822fa1374cad_8d070e80ed8a54f8] with query id 51fc822fa1374cad-8d070e80ed8a54f8 and txn id 5725
RuntimeLogger 2025-03-24 11:03:35,314 INFO (mysql-nio-pool-44|6692) [Coordinator.execInternal():776] dispatch load job: 51fc822fa1374cad-8d070e80ed8a54f8 to [TNetworkAddress(hostname:172.20.80.5, port:9060)]
RuntimeLogger 2025-03-24 11:03:35,315 INFO (mysql-nio-pool-44|6692) [Coordinator.sendPipelineCtx():1038] num local sink for backend 10004 is 1
RuntimeLogger 2025-03-24 11:03:35,320 INFO (mysql-nio-pool-42|6362) [StmtExecutor.handleQueryStmt():1827] Query 4ed91d6c734440a4-b4c500e6e9d02717 finished
RuntimeLogger 2025-03-24 11:03:35,377 INFO (mysql-nio-pool-44|6692) [DatabaseTransactionMgr.commitTransaction():828] transaction:[TransactionState. transaction id: 5725, label: label_51fc822fa1374cad_8d070e80ed8a54f8, db id: 10162, table id list: 11468, callback id: -1, coordinator: FE: 172.20.80.22, transaction status: COMMITTED, error replicas num: 0, replica ids: , prepare time: 1742814215312, commit time: 1742814215373, finish time: -1, reason: ] successfully committed
RuntimeLogger 2025-03-24 11:03:35,377 INFO (PUBLISH_VERSION|20) [PublishVersionDaemon.genPublishTask():178] send publish tasks for transaction: 5725, db: 10162
RuntimeLogger 2025-03-24 11:03:35,392 INFO (PUBLISH_VERSION|20) [OlapTable.updateVisibleVersionAndTime():2984] updateVisibleVersionAndTime, tableName: audit_operation_logs, visibleVersion, 3714, visibleVersionTime: 1742814215388
RuntimeLogger 2025-03-24 11:03:35,395 INFO (PUBLISH_VERSION|20) [MTMVService.processEvent():185] processEvent, Event: TableEvent{ctlId=0, ctlName='internal', dbId=10162, dbName='doris_db', tableId=11468, tableName='audit_operation_logs'} Event{eventId=33532, eventTime=1742814215395, eventType=DATA_CHANGE}
RuntimeLogger 2025-03-24 11:03:35,396 INFO (PUBLISH_VERSION|20) [DatabaseTransactionMgr.finishTransaction():1110] finish transaction TransactionState. transaction id: 5725, label: label_51fc822fa1374cad_8d070e80ed8a54f8, db id: 10162, table id list: 11468, callback id: -1, coordinator: FE: 172.20.80.22, transaction status: VISIBLE, error replicas num: 0, replica ids: , prepare time: 1742814215312, commit time: 1742814215373, finish time: 1742814215388, reason:  successfully, publish times 1, publish result QUORUM_SUCC
RuntimeLogger 2025-03-24 11:03:35,473 INFO (mysql-nio-pool-44|6692) [Coordinator.execInternal():756] dispatch result sink of query ead4d5064eb4455-88e0d473fc662532 to TNetworkAddress(hostname:172.20.80.7, port:9060)
RuntimeLogger 2025-03-24 11:03:35,488 INFO (mysql-nio-pool-44|6692) [StmtExecutor.handleQueryStmt():1827] Query ead4d5064eb4455-88e0d473fc662532 finished
RuntimeLogger 2025-03-24 11:03:35,525 INFO (mysql-nio-pool-44|6692) [Coordinator.execInternal():756] dispatch result sink of query b3f6e298bd224228-a89566627c54434b to TNetworkAddress(hostname:172.20.80.6, port:9060)
RuntimeLogger 2025-03-24 11:03:35,550 INFO (mysql-nio-pool-44|6692) [StmtExecutor.handleQueryStmt():1827] Query b3f6e298bd224228-a89566627c54434b finished
RuntimeLogger 2025-03-24 11:03:36,037 INFO (binlog-gcer|48) [BinlogManager.gc():401] begin gc binlog
RuntimeLogger 2025-03-24 11:03:36,037 INFO (binlog-gcer|48) [BinlogManager.gc():412] gc binlog, dbBinlogMap is null
RuntimeLogger 2025-03-24 11:03:36,037 INFO (binlog-gcer|48) [BinlogGcer.runAfterCatalogReady():63] no gc binlog
RuntimeLogger 2025-03-24 11:03:37,572 INFO (mysql-nio-pool-44|6692) [Coordinator.execInternal():756] dispatch result sink of query cc38c75223a14005-92e9771cd9ed280e to TNetworkAddress(hostname:172.20.80.7, port:9060)
RuntimeLogger 2025-03-24 11:03:37,573 INFO (mysql-nio-pool-42|6362) [Coordinator.execInternal():756] dispatch result sink of query 1c3f1f0999fc4ccc-aa1bade62fe22726 to TNetworkAddress(hostname:172.20.80.5, port:9060)
RuntimeLogger 2025-03-24 11:03:37,597 INFO (mysql-nio-pool-44|6692) [StmtExecutor.handleQueryStmt():1827] Query cc38c75223a14005-92e9771cd9ed280e finished
RuntimeLogger 2025-03-24 11:03:37,597 INFO (mysql-nio-pool-42|6362) [StmtExecutor.handleQueryStmt():1827] Query 1c3f1f0999fc4ccc-aa1bade62fe22726 finished
RuntimeLogger 2025-03-24 11:03:37,739 INFO (mysql-nio-pool-42|6362) [Coordinator.execInternal():756] dispatch result sink of query 158bcd7d1af84f5d-98dbd9769e13f98d to TNetworkAddress(hostname:172.20.80.7, port:9060)
RuntimeLogger 2025-03-24 11:03:37,751 INFO (mysql-nio-pool-44|6692) [Coordinator.execInternal():756] dispatch result sink of query 7788aa7db7d84dad-bad990781ca0a9eb to TNetworkAddress(hostname:172.20.80.6, port:9060)
RuntimeLogger 2025-03-24 11:03:37,754 INFO (mysql-nio-pool-42|6362) [StmtExecutor.handleQueryStmt():1827] Query 158bcd7d1af84f5d-98dbd9769e13f98d finished
RuntimeLogger 2025-03-24 11:03:37,769 INFO (mysql-nio-pool-42|6362) [Coordinator.execInternal():756] dispatch result sink of query 20a7dfad245c4a2d-8e53c1232f492901 to TNetworkAddress(hostname:172.20.80.5, port:9060)
RuntimeLogger 2025-03-24 11:03:37,773 INFO (mysql-nio-pool-44|6692) [StmtExecutor.handleQueryStmt():1827] Query 7788aa7db7d84dad-bad990781ca0a9eb finished
RuntimeLogger 2025-03-24 11:03:37,783 INFO (mysql-nio-pool-42|6362) [StmtExecutor.handleQueryStmt():1827] Query 20a7dfad245c4a2d-8e53c1232f492901 finished
RuntimeLogger 2025-03-24 11:03:37,788 INFO (mysql-nio-pool-44|6692) [Coordinator.execInternal():756] dispatch result sink of query e432bf185af343b0-9f9e24b37c8fe170 to TNetworkAddress(hostname:172.20.80.7, port:9060)
RuntimeLogger 2025-03-24 11:03:37,849 INFO (mysql-nio-pool-44|6692) [StmtExecutor.handleQueryStmt():1827] Query e432bf185af343b0-9f9e24b37c8fe170 finished
RuntimeLogger 2025-03-24 11:03:37,989 INFO (mysql-nio-pool-44|6692) [DatabaseTransactionMgr.beginTransaction():397] begin transaction: txn id 5726 with label label_5094b552cc804742_a6fc243eec955356 from coordinator FE: 172.20.80.22, listener id: -1
RuntimeLogger 2025-03-24 11:03:37,990 INFO (mysql-nio-pool-44|6692) [OlapInsertExecutor.beforeExec():197] start insert [label_5094b552cc804742_a6fc243eec955356] with query id 5094b552cc804742-a6fc243eec955356 and txn id 5726
RuntimeLogger 2025-03-24 11:03:37,990 INFO (mysql-nio-pool-44|6692) [Coordinator.execInternal():776] dispatch load job: 5094b552cc804742-a6fc243eec955356 to [TNetworkAddress(hostname:172.20.80.6, port:9060)]
RuntimeLogger 2025-03-24 11:03:37,991 INFO (mysql-nio-pool-44|6692) [Coordinator.sendPipelineCtx():1038] num local sink for backend 10006 is 1
RuntimeLogger 2025-03-24 11:03:38,047 INFO (mysql-nio-pool-44|6692) [DatabaseTransactionMgr.commitTransaction():828] transaction:[TransactionState. transaction id: 5726, label: label_5094b552cc804742_a6fc243eec955356, db id: 10162, table id list: 11468, callback id: -1, coordinator: FE: 172.20.80.22, transaction status: COMMITTED, error replicas num: 0, replica ids: , prepare time: 1742814217988, commit time: 1742814218044, finish time: -1, reason: ] successfully committed
RuntimeLogger 2025-03-24 11:03:38,054 INFO (PUBLISH_VERSION|20) [PublishVersionDaemon.genPublishTask():178] send publish tasks for transaction: 5726, db: 10162
RuntimeLogger 2025-03-24 11:03:38,067 INFO (PUBLISH_VERSION|20) [OlapTable.updateVisibleVersionAndTime():2984] updateVisibleVersionAndTime, tableName: audit_operation_logs, visibleVersion, 3715, visibleVersionTime: 1742814218064
RuntimeLogger 2025-03-24 11:03:38,071 INFO (PUBLISH_VERSION|20) [MTMVService.processEvent():185] processEvent, Event: TableEvent{ctlId=0, ctlName='internal', dbId=10162, dbName='doris_db', tableId=11468, tableName='audit_operation_logs'} Event{eventId=33534, eventTime=1742814218071, eventType=DATA_CHANGE}
RuntimeLogger 2025-03-24 11:03:38,071 INFO (PUBLISH_VERSION|20) [DatabaseTransactionMgr.finishTransaction():1110] finish transaction TransactionState. transaction id: 5726, label: label_5094b552cc804742_a6fc243eec955356, db id: 10162, table id list: 11468, callback id: -1, coordinator: FE: 172.20.80.22, transaction status: VISIBLE, error replicas num: 0, replica ids: , prepare time: 1742814217988, commit time: 1742814218044, finish time: 1742814218064, reason:  successfully, publish times 1, publish result QUORUM_SUCC
RuntimeLogger 2025-03-24 11:03:38,178 INFO (mysql-nio-pool-44|6692) [DatabaseTransactionMgr.beginTransaction():397] begin transaction: txn id 5727 with label label_561988f831964eb0_a11e1c4d694f2861 from coordinator FE: 172.20.80.22, listener id: -1
RuntimeLogger 2025-03-24 11:03:38,179 INFO (mysql-nio-pool-44|6692) [OlapInsertExecutor.beforeExec():197] start insert [label_561988f831964eb0_a11e1c4d694f2861] with query id 561988f831964eb0-a11e1c4d694f2861 and txn id 5727
RuntimeLogger 2025-03-24 11:03:38,179 INFO (mysql-nio-pool-44|6692) [Coordinator.execInternal():776] dispatch load job: 561988f831964eb0-a11e1c4d694f2861 to [TNetworkAddress(hostname:172.20.80.5, port:9060)]
RuntimeLogger 2025-03-24 11:03:38,180 INFO (mysql-nio-pool-44|6692) [Coordinator.sendPipelineCtx():1038] num local sink for backend 10004 is 1
RuntimeLogger 2025-03-24 11:03:38,233 INFO (mysql-nio-pool-44|6692) [DatabaseTransactionMgr.commitTransaction():828] transaction:[TransactionState. transaction id: 5727, label: label_561988f831964eb0_a11e1c4d694f2861, db id: 10162, table id list: 11468, callback id: -1, coordinator: FE: 172.20.80.22, transaction status: COMMITTED, error replicas num: 0, replica ids: , prepare time: 1742814218178, commit time: 1742814218229, finish time: -1, reason: ] successfully committed
RuntimeLogger 2025-03-24 11:03:38,234 INFO (PUBLISH_VERSION|20) [PublishVersionDaemon.genPublishTask():178] send publish tasks for transaction: 5727, db: 10162
RuntimeLogger 2025-03-24 11:03:38,248 INFO (PUBLISH_VERSION|20) [OlapTable.updateVisibleVersionAndTime():2984] updateVisibleVersionAndTime, tableName: audit_operation_logs, visibleVersion, 3716, visibleVersionTime: 1742814218245
RuntimeLogger 2025-03-24 11:03:38,252 INFO (PUBLISH_VERSION|20) [MTMVService.processEvent():185] processEvent, Event: TableEvent{ctlId=0, ctlName='internal', dbId=10162, dbName='doris_db', tableId=11468, tableName='audit_operation_logs'} Event{eventId=33536, eventTime=1742814218251, eventType=DATA_CHANGE}
RuntimeLogger 2025-03-24 11:03:38,252 INFO (PUBLISH_VERSION|20) [DatabaseTransactionMgr.finishTransaction():1110] finish transaction TransactionState. transaction id: 5727, label: label_561988f831964eb0_a11e1c4d694f2861, db id: 10162, table id list: 11468, callback id: -1, coordinator: FE: 172.20.80.22, transaction status: VISIBLE, error replicas num: 0, replica ids: , prepare time: 1742814218178, commit time: 1742814218229, finish time: 1742814218245, reason:  successfully, publish times 1, publish result QUORUM_SUCC
RuntimeLogger 2025-03-24 11:03:38,457 INFO (mysql-nio-pool-44|6692) [DatabaseTransactionMgr.beginTransaction():397] begin transaction: txn id 5728 with label label_75cca83f69cc446d_90351dea77f31c53 from coordinator FE: 172.20.80.22, listener id: -1
RuntimeLogger 2025-03-24 11:03:38,458 INFO (mysql-nio-pool-44|6692) [OlapInsertExecutor.beforeExec():197] start insert [label_75cca83f69cc446d_90351dea77f31c53] with query id 75cca83f69cc446d-90351dea77f31c53 and txn id 5728
RuntimeLogger 2025-03-24 11:03:38,458 INFO (mysql-nio-pool-44|6692) [Coordinator.execInternal():776] dispatch load job: 75cca83f69cc446d-90351dea77f31c53 to [TNetworkAddress(hostname:172.20.80.7, port:9060)]
RuntimeLogger 2025-03-24 11:03:38,458 INFO (mysql-nio-pool-44|6692) [Coordinator.sendPipelineCtx():1038] num local sink for backend 10005 is 1
RuntimeLogger 2025-03-24 11:03:38,515 INFO (mysql-nio-pool-44|6692) [DatabaseTransactionMgr.commitTransaction():828] transaction:[TransactionState. transaction id: 5728, label: label_75cca83f69cc446d_90351dea77f31c53, db id: 10162, table id list: 11468, callback id: -1, coordinator: FE: 172.20.80.22, transaction status: COMMITTED, error replicas num: 0, replica ids: , prepare time: 1742814218457, commit time: 1742814218511, finish time: -1, reason: ] successfully committed
RuntimeLogger 2025-03-24 11:03:38,516 INFO (PUBLISH_VERSION|20) [PublishVersionDaemon.genPublishTask():178] send publish tasks for transaction: 5728, db: 10162
RuntimeLogger 2025-03-24 11:03:38,531 INFO (PUBLISH_VERSION|20) [OlapTable.updateVisibleVersionAndTime():2984] updateVisibleVersionAndTime, tableName: audit_operation_logs, visibleVersion, 3717, visibleVersionTime: 1742814218527
RuntimeLogger 2025-03-24 11:03:38,534 INFO (PUBLISH_VERSION|20) [MTMVService.processEvent():185] processEvent, Event: TableEvent{ctlId=0, ctlName='internal', dbId=10162, dbName='doris_db', tableId=11468, tableName='audit_operation_logs'} Event{eventId=33538, eventTime=1742814218534, eventType=DATA_CHANGE}
RuntimeLogger 2025-03-24 11:03:38,534 INFO (PUBLISH_VERSION|20) [DatabaseTransactionMgr.finishTransaction():1110] finish transaction TransactionState. transaction id: 5728, label: label_75cca83f69cc446d_90351dea77f31c53, db id: 10162, table id list: 11468, callback id: -1, coordinator: FE: 172.20.80.22, transaction status: VISIBLE, error replicas num: 0, replica ids: , prepare time: 1742814218457, commit time: 1742814218511, finish time: 1742814218527, reason:  successfully, publish times 1, publish result QUORUM_SUCC
RuntimeLogger 2025-03-24 11:03:38,627 INFO (mysql-nio-pool-44|6692) [DatabaseTransactionMgr.beginTransaction():397] begin transaction: txn id 5729 with label label_a40636a55e69454d_aaeadc94bcf6042b from coordinator FE: 172.20.80.22, listener id: -1
RuntimeLogger 2025-03-24 11:03:38,628 INFO (mysql-nio-pool-44|6692) [OlapInsertExecutor.beforeExec():197] start insert [label_a40636a55e69454d_aaeadc94bcf6042b] with query id a40636a55e69454d-aaeadc94bcf6042b and txn id 5729
RuntimeLogger 2025-03-24 11:03:38,628 INFO (mysql-nio-pool-44|6692) [Coordinator.execInternal():776] dispatch load job: a40636a55e69454d-aaeadc94bcf6042b to [TNetworkAddress(hostname:172.20.80.6, port:9060)]
RuntimeLogger 2025-03-24 11:03:38,629 INFO (mysql-nio-pool-44|6692) [Coordinator.sendPipelineCtx():1038] num local sink for backend 10006 is 1
RuntimeLogger 2025-03-24 11:03:38,685 INFO (mysql-nio-pool-44|6692) [DatabaseTransactionMgr.commitTransaction():828] transaction:[TransactionState. transaction id: 5729, label: label_a40636a55e69454d_aaeadc94bcf6042b, db id: 10162, table id list: 11468, callback id: -1, coordinator: FE: 172.20.80.22, transaction status: COMMITTED, error replicas num: 0, replica ids: , prepare time: 1742814218627, commit time: 1742814218680, finish time: -1, reason: ] successfully committed
RuntimeLogger 2025-03-24 11:03:38,688 INFO (PUBLISH_VERSION|20) [PublishVersionDaemon.genPublishTask():178] send publish tasks for transaction: 5729, db: 10162
RuntimeLogger 2025-03-24 11:03:38,702 INFO (PUBLISH_VERSION|20) [OlapTable.updateVisibleVersionAndTime():2984] updateVisibleVersionAndTime, tableName: audit_operation_logs, visibleVersion, 3718, visibleVersionTime: 1742814218698
RuntimeLogger 2025-03-24 11:03:38,706 INFO (PUBLISH_VERSION|20) [MTMVService.processEvent():185] processEvent, Event: TableEvent{ctlId=0, ctlName='internal', dbId=10162, dbName='doris_db', tableId=11468, tableName='audit_operation_logs'} Event{eventId=33540, eventTime=1742814218705, eventType=DATA_CHANGE}
RuntimeLogger 2025-03-24 11:03:38,706 INFO (PUBLISH_VERSION|20) [DatabaseTransactionMgr.finishTransaction():1110] finish transaction TransactionState. transaction id: 5729, label: label_a40636a55e69454d_aaeadc94bcf6042b, db id: 10162, table id list: 11468, callback id: -1, coordinator: FE: 172.20.80.22, transaction status: VISIBLE, error replicas num: 0, replica ids: , prepare time: 1742814218627, commit time: 1742814218680, finish time: 1742814218698, reason:  successfully, publish times 1, publish result QUORUM_SUCC

be.gc.log 崩溃前日志

OpenJDK 64-Bit Server VM (25.352-b11) for linux-aarch64 JRE (1.8.0_352-b11), built on Nov 10 2022 10:27:39 by "root" with gcc 7.3.0
Memory: 64k page, physical 9437184k(9070720k free), swap 0k(0k free)
CommandLine flags: -XX:-CriticalJNINatives -XX:InitialHeapSize=150994944 -XX:MaxHeapSize=2147483648 -XX:+PrintGC -XX:+PrintGCTimeStamps -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseParallelGC 
1.651: [GC (Allocation Failure)  40960K->6064K(157184K), 0.0145441 secs]
2.031: [GC (Allocation Failure)  47024K->4005K(157184K), 0.0146940 secs]
2.659: [GC (Metadata GC Threshold)  41592K->11638K(157184K), 0.0130492 secs]
2.673: [Full GC (Metadata GC Threshold)  11638K->11254K(125440K), 0.0588623 secs]
2.980: [GC (Allocation Failure)  52214K->14154K(141824K), 0.0038371 secs]
3.106: [GC (Allocation Failure)  75082K->11685K(162816K), 0.0038224 secs]
3.232: [GC (Allocation Failure)  90021K->11454K(182784K), 0.0035067 secs]
3.337: [GC (Allocation Failure)  103102K->11454K(182784K), 0.0034706 secs]
3.441: [GC (Allocation Failure)  103102K->11422K(231936K), 0.0032081 secs]
3.643: [GC (Allocation Failure)  152734K->11568K(231424K), 0.0026882 secs]
3.797: [GC (Allocation Failure)  152880K->11488K(271360K), 0.0027893 secs]

be.warning.log 崩溃前日志

RuntimeLogger W20250324 19:03:26.494997  1482 status.h:413] meet error status: [TIMEOUT]Query tiemout

	0#  doris::ResultBufferMgr::cancel_thread()
	1#  doris::Thread::supervise_thread(void*)
	2#  ?
	3#  ?
RuntimeLogger W20250324 19:03:27.495492  1482 status.h:413] meet error status: [TIMEOUT]Query tiemout

	0#  doris::ResultBufferMgr::cancel_thread()
	1#  doris::Thread::supervise_thread(void*)
	2#  ?
	3#  ?
RuntimeLogger W20250324 19:03:27.495751  1482 status.h:413] meet error status: [TIMEOUT]Query tiemout

	0#  doris::ResultBufferMgr::cancel_thread()
	1#  doris::Thread::supervise_thread(void*)
	2#  ?
	3#  ?
RuntimeLogger W20250324 19:03:33.496768  1482 status.h:413] meet error status: [TIMEOUT]Query tiemout

	0#  doris::ResultBufferMgr::cancel_thread()
	1#  doris::Thread::supervise_thread(void*)
	2#  ?
	3#  ?
RuntimeLogger W20250324 19:03:43.498467  1482 status.h:413] meet error status: [TIMEOUT]Query tiemout

	0#  doris::ResultBufferMgr::cancel_thread()
	1#  doris::Thread::supervise_thread(void*)
	2#  ?
	3#  ?

be.out

be.put
```log
INFO: java_cmd /usr/lib/jvm/java/bin/java
INFO: jdk_version 8
INFO: java_cmd /usr/lib/jvm/java/bin/java
INFO: jdk_version 8
INFO: java_cmd /usr/lib/jvm/java/bin/java
INFO: jdk_version 8
INFO: java_cmd /usr/lib/jvm/java/bin/java
INFO: jdk_version 8
INFO: java_cmd /usr/lib/jvm/java/bin/java
INFO: jdk_version 8
2 Answers

be.info.log 崩溃前的日志

RuntimeLogger I20250324 19:03:38.520237  1746 engine_publish_version_task.cpp:425] publish version successfully on tablet, table_id=11468, tablet=11490, transaction_id=5728, version=3717, num_rows=0, res=[OK], cost: 1446(us) 
RuntimeLogger I20250324 19:03:38.521312  1747 engine_publish_version_task.cpp:425] publish version successfully on tablet, table_id=11468, tablet=11498, transaction_id=5728, version=3717, num_rows=0, res=[OK], cost: 2429(us) 
RuntimeLogger I20250324 19:03:38.521095  1739 tablet.cpp:3686] [Publish] construct delete bitmap tablet: 11494, rowset_ids to add: 0, rowset_ids to del: 0, cur version: 3717, transaction_id: 5728,cost: 1784(us) , total rows: 0
RuntimeLogger I20250324 19:03:38.520177  1744 tablet.cpp:3686] [Publish] construct delete bitmap tablet: 11486, rowset_ids to add: 0, rowset_ids to del: 0, cur version: 3717, transaction_id: 5728,cost: 757(us) , total rows: 1
RuntimeLogger I20250324 19:03:38.521247  1745 tablet.cpp:3686] [Publish] construct delete bitmap tablet: 11482, rowset_ids to add: 0, rowset_ids to del: 0, cur version: 3717, transaction_id: 5728,cost: 2106(us) , total rows: 0
RuntimeLogger I20250324 19:03:38.522393  1739 engine_publish_version_task.cpp:425] publish version successfully on tablet, table_id=11468, tablet=11494, transaction_id=5728, version=3717, num_rows=0, res=[OK], cost: 3582(us) 
RuntimeLogger I20250324 19:03:38.522702  1745 engine_publish_version_task.cpp:425] publish version successfully on tablet, table_id=11468, tablet=11482, transaction_id=5728, version=3717, num_rows=0, res=[OK], cost: 3970(us) 
RuntimeLogger I20250324 19:03:38.522754  1744 engine_publish_version_task.cpp:425] publish version successfully on tablet, table_id=11468, tablet=11486, transaction_id=5728, version=3717, num_rows=1, res=[OK], cost: 3998(us) 
RuntimeLogger I20250324 19:03:38.523105  1841 engine_publish_version_task.cpp:346] finish to publish version on transaction.transaction_id=5728, cost(us): 4718, error_tablet_size=0, res=[OK]
RuntimeLogger I20250324 19:03:38.523258  1841 task_worker_pool.cpp:1614] successfully publish version|signature=5728|transaction_id=5728|tablets_num=8|cost(s)=0
RuntimeLogger I20250324 19:03:38.536307  3113 task_worker_pool.cpp:337] successfully submit task|type=UPDATE_VISIBLE_VERSION|signature=-1
RuntimeLogger I20250324 19:03:38.641419  2101 internal_service.cpp:396] open load stream, load_id=a40636a55e69454d-aaeadc94bcf6042b, src_id=10006
RuntimeLogger I20250324 19:03:38.641796  2101 load_stream.cpp:423] succeed to init load stream load_id=a40636a55e69454d-aaeadc94bcf6042b, txn_id=5729
RuntimeLogger I20250324 19:03:38.643630  2113 internal_service.cpp:396] open load stream, load_id=a40636a55e69454d-aaeadc94bcf6042b, src_id=10006
RuntimeLogger I20250324 19:03:38.653513  2291 load_stream.cpp:438] received CLOSE_LOAD from sender 10006, remaining 1 senders, load_id=a40636a55e69454d-aaeadc94bcf6042b, txn_id=5729
RuntimeLogger I20250324 19:03:38.653851  2284 load_stream.cpp:459] report result load_id=a40636a55e69454d-aaeadc94bcf6042b, txn_id=5729, success tablet num 0, failed tablet num 0
RuntimeLogger I20250324 19:03:38.654282  2301 load_stream.cpp:662] stream 11717 on_closed, remaining streams = 1, load_id=a40636a55e69454d-aaeadc94bcf6042b, txn_id=5729
RuntimeLogger I20250324 19:03:38.662740  2376 load_stream.cpp:438] received CLOSE_LOAD from sender 10006, remaining 0 senders, load_id=a40636a55e69454d-aaeadc94bcf6042b, txn_id=5729
RuntimeLogger I20250324 19:03:38.665710  1971 rowset_builder.cpp:287] submit calc delete bitmap task to executor, tablet_id: 11498, txn_id: 5729
RuntimeLogger I20250324 19:03:38.665977  1971 tablet.cpp:3186] skip to construct delete bitmap tablet: 11498 rowset: 020000000000c6dd2d48229be05d25878b571f59d3c6e7bd
RuntimeLogger I20250324 19:03:38.666052  1971 tablet.cpp:3538] [Before Commit] construct delete bitmap tablet: 11498, rowset_ids to add: 0, rowset_ids to del: 0, cur max_version: 3717, transaction_id: 5729, total rows: 0
RuntimeLogger I20250324 19:03:38.666093  1971 rowset_builder.cpp:301] Got result of calc delete bitmap task from executor, tablet_id: 11498, txn_id: 5729
RuntimeLogger I20250324 19:03:38.666803  1988 rowset_builder.cpp:287] submit calc delete bitmap task to executor, tablet_id: 11494, txn_id: 5729
RuntimeLogger I20250324 19:03:38.666958  1988 tablet.cpp:3186] skip to construct delete bitmap tablet: 11494 rowset: 020000000000c6dc2d48229be05d25878b571f59d3c6e7bd
RuntimeLogger I20250324 19:03:38.666985  1988 tablet.cpp:3538] [Before Commit] construct delete bitmap tablet: 11494, rowset_ids to add: 0, rowset_ids to del: 0, cur max_version: 3717, transaction_id: 5729, total rows: 0
RuntimeLogger I20250324 19:03:38.667011  1988 rowset_builder.cpp:301] Got result of calc delete bitmap task from executor, tablet_id: 11494, txn_id: 5729
RuntimeLogger I20250324 19:03:38.667505  1467 load_stream_writer.cpp:156] segment 0 path /opt/apache-doris/be/storage/data/75/11482/693420982/020000000000c6d62d48229be05d25878b571f59d3c6e7bd_0.datclosed, written 8324 bytes
RuntimeLogger I20250324 19:03:38.667636  1973 rowset_builder.cpp:287] submit calc delete bitmap task to executor, tablet_id: 11490, txn_id: 5729
RuntimeLogger I20250324 19:03:38.667832  1973 tablet.cpp:3186] skip to construct delete bitmap tablet: 11490 rowset: 020000000000c6db2d48229be05d25878b571f59d3c6e7bd
RuntimeLogger I20250324 19:03:38.667918  1973 tablet.cpp:3538] [Before Commit] construct delete bitmap tablet: 11490, rowset_ids to add: 0, rowset_ids to del: 0, cur max_version: 3717, transaction_id: 5729, total rows: 0
RuntimeLogger I20250324 19:03:38.667955  1973 rowset_builder.cpp:301] Got result of calc delete bitmap task from executor, tablet_id: 11490, txn_id: 5729
RuntimeLogger I20250324 19:03:38.668422  1467 tablet.cpp:3167] calc segment delete bitmap, tablet: 11482 rowset: 020000000000c6d62d48229be05d25878b571f59d3c6e7bd seg_id: 0 dummy_version: 3718 rows: 1 conflict rows: 0 bimap num: 22 cost: 491(us)
RuntimeLogger I20250324 19:03:38.668563  1983 rowset_builder.cpp:287] submit calc delete bitmap task to executor, tablet_id: 11486, txn_id: 5729
RuntimeLogger I20250324 19:03:38.668942  1983 tablet.cpp:3186] skip to construct delete bitmap tablet: 11486 rowset: 020000000000c6da2d48229be05d25878b571f59d3c6e7bd
RuntimeLogger I20250324 19:03:38.669000  1983 tablet.cpp:3538] [Before Commit] construct delete bitmap tablet: 11486, rowset_ids to add: 0, rowset_ids to del: 0, cur max_version: 3717, transaction_id: 5729, total rows: 0
RuntimeLogger I20250324 19:03:38.669032  1983 rowset_builder.cpp:301] Got result of calc delete bitmap task from executor, tablet_id: 11486, txn_id: 5729
RuntimeLogger I20250324 19:03:38.668591  1467 beta_rowset_writer.cpp:192] [Memtable Flush] construct delete bitmap tablet: 11482, rowset_ids: 22, cur max_version: 3717, transaction_id: 5729, cost: 845(us), total rows: 1
RuntimeLogger I20250324 19:03:38.669824  1957 rowset_builder.cpp:287] submit calc delete bitmap task to executor, tablet_id: 11478, txn_id: 5729
RuntimeLogger I20250324 19:03:38.670151  1957 tablet.cpp:3186] skip to construct delete bitmap tablet: 11478 rowset: 020000000000c6d92d48229be05d25878b571f59d3c6e7bd
RuntimeLogger I20250324 19:03:38.670194  1957 tablet.cpp:3538] [Before Commit] construct delete bitmap tablet: 11478, rowset_ids to add: 0, rowset_ids to del: 0, cur max_version: 3717, transaction_id: 5729, total rows: 0
RuntimeLogger I20250324 19:03:38.670226  1957 rowset_builder.cpp:301] Got result of calc delete bitmap task from executor, tablet_id: 11478, txn_id: 5729
RuntimeLogger I20250324 19:03:38.670884  1968 rowset_builder.cpp:287] submit calc delete bitmap task to executor, tablet_id: 11474, txn_id: 5729
RuntimeLogger I20250324 19:03:38.671173  1968 tablet.cpp:3186] skip to construct delete bitmap tablet: 11474 rowset: 020000000000c6d82d48229be05d25878b571f59d3c6e7bd
RuntimeLogger I20250324 19:03:38.671217  1968 tablet.cpp:3538] [Before Commit] construct delete bitmap tablet: 11474, rowset_ids to add: 0, rowset_ids to del: 0, cur max_version: 3717, transaction_id: 5729, total rows: 0
RuntimeLogger I20250324 19:03:38.671252  1968 rowset_builder.cpp:301] Got result of calc delete bitmap task from executor, tablet_id: 11474, txn_id: 5729
RuntimeLogger I20250324 19:03:38.671911  1987 rowset_builder.cpp:287] submit calc delete bitmap task to executor, tablet_id: 11470, txn_id: 5729
RuntimeLogger I20250324 19:03:38.672169  1987 tablet.cpp:3186] skip to construct delete bitmap tablet: 11470 rowset: 020000000000c6d72d48229be05d25878b571f59d3c6e7bd
RuntimeLogger I20250324 19:03:38.672215  1987 tablet.cpp:3538] [Before Commit] construct delete bitmap tablet: 11470, rowset_ids to add: 0, rowset_ids to del: 0, cur max_version: 3717, transaction_id: 5729, total rows: 0
RuntimeLogger I20250324 19:03:38.672251  1987 rowset_builder.cpp:301] Got result of calc delete bitmap task from executor, tablet_id: 11470, txn_id: 5729
RuntimeLogger I20250324 19:03:38.673213  1965 rowset_builder.cpp:287] submit calc delete bitmap task to executor, tablet_id: 11482, txn_id: 5729
RuntimeLogger I20250324 19:03:38.673430  1965 tablet.cpp:3186] skip to construct delete bitmap tablet: 11482 rowset: 020000000000c6d62d48229be05d25878b571f59d3c6e7bd
RuntimeLogger I20250324 19:03:38.673468  1965 tablet.cpp:3538] [Before Commit] construct delete bitmap tablet: 11482, rowset_ids to add: 0, rowset_ids to del: 0, cur max_version: 3717, transaction_id: 5729, total rows: 1
RuntimeLogger I20250324 19:03:38.673568  1965 r

如果be 最后挂了,需要贴一下be.out。 如果be 没挂,但是cpu 很高,可以top -H 看看