BE报错Check failed: block.rows() == tmp_block.rows()导致BE进程退出

Viewed 136

版本:1.2.7

从一张表迁移数据到另外一张表,其中a和b表结构相同,40个字段

insert into
  b
select
  *
from
  a where date = 'xxx';

be.WARNING日志

I0328 18:27:00.379040 83192 tablets_channel.cpp:102] close tablets channel: (id=0748e4da2a38b20a-6bc3e0ec47ae6fa8,index_id=127899), sender id: 0, backend id: 10005
I0328 18:27:00.394711 82174 daemon.cpp:233] OS physical memory 187.41 GB. Process memory usage 48.80 GB, limit 149.93 GB, soft limit 134.94 GB. Sys available memory 127.57 GB, low water mark 1.60 GB, warning water mark 3.20 GB. Refresh interval memory growth 0 B
I0328 18:27:00.399092 83349 tablet_sink.cpp:1489] all node channels are stopped(maybe finished/offending/cancelled), sender thread exit. 748e4da2a38b20a-6bc3e0ec47ae6fa8
I0328 18:27:00.417482 83192 load_channel.cpp:44] load channel removed. mem peak usage=11000694, info=label: LoadChannel#senderIp=xxx#loadID=0748e4da2a38b20a-6bc3e0ec47ae6fa8; consumption: 11000694; peak_consumption: 11000694; , load_id=0748e4da2a38b20a-6bc3e0ec47ae6fa8, is high priority=0, sender_ip=xxx, is_vec=1
F0328 18:27:00.420957 82302 vtablet_sink.cpp:305] Check failed: block.rows() == tmp_block.rows()
*** Check failure stack trace: ***
@ 0x5580f3572cad google::LogMessage::Fail()
@ 0x5580f35751e9 google::LogMessage::SendToLog()
@ 0x5580f3572816 google::LogMessage::Flush()
@ 0x5580f3575859 google::LogMessageFatal::~LogMessageFatal()
I0328 18:27:00.447638 120874 tablet_sink.cpp:1207] total mem_exceeded_block_ns=0, total queue_push_lock_ns=27074, total actual_consume_ns=49489303, load id=748e4da2a38b20a-6bc3e0ec47ae6fa8
I0328 18:27:00.447690 120874 tablet_sink.cpp:1241] finished to close olap table sink. load_id=748e4da2a38b20a-6bc3e0ec47ae6fa8, txn_id=79016616, node add batch time(ms)/wait execution time(ms)/close time(ms)/num: {10004:(66)(0)(122)(2)} {10006:(8)(0)(122)(1)} {10007:(6)(0)(122)(1)} {10003:(72)(0)(122)(2)} {10005:(64)(0)(92)(2)}
@ 0x5580f2f0436b doris::stream_load::VNodeChannel::try_send_block()
I0328 18:27:00.452757 120874 plan_fragment_executor.cpp:715] Close() fragment_instance_id=748e4da2a38b20a-6bc3e0ec47ae6fa9
@ 0x5580ee3af945 doris::ThreadPool::dispatch_thread()
@ 0x5580ee3a4d9f doris::Thread::supervise_thread()
I0328 18:27:00.456377 120874 query_fragments_ctx.h:56] Deregister query/load memory tracker, queryId=748e4da2a38b20a-6bc3e0ec47ae6fa8, Limit=2.00 GB, CurrUsed=4.86 MB, PeakUsed=34.02 MB
I0328 18:27:00.457376 131071 task_worker_pool.cpp:249] successfully submit task|type=PUBLISH_VERSION|signature=79016616|queue_size=1
I0328 18:27:00.457677 83057 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=127898, tablet=377608.1085535287.7b44e46de53d395b-e5230029b6aa3ba2, transaction_id=79016616, version=9296, num_rows=0, res=[OK]
I0328 18:27:00.457773 83050 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=127898, tablet=377616.1085535287.cc408598d3f6b439-df77907890ed9b94, transaction_id=79016616, version=9296, num_rows=0, res=[OK]
I0328 18:27:00.457787 83037 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=127898, tablet=377620.1085535287.a6448120ce8ff302-5a4210deec129fa8, transaction_id=79016616, version=9296, num_rows=10119, res=[OK]
I0328 18:27:00.457834 83083 engine_publish_version_task.cpp:190] finish to publish version on transaction.transaction_id=79016616, cost(us): 442, error_tablet_size=0, res=[OK]
I0328 18:27:00.457859 83083 task_worker_pool.cpp:727] successfully publish version|signature=79016616|transaction_id=79016616|tablets_num=3
@ 0x7fb2707f1ea5 start_thread
I0328 18:27:00.467916 83387 stream_load.cpp:634] put stream_load_record rocksdb successfully. label: flink_connector_20240328_182700_4ac27bff605f48eaa619c6f3666574e7, key: 1711621620467_flink_connector_20240328_182700_4ac27bff605f48eaa619c6f3666574e7
@ 0x7fb270b0496d __clone
@ (nil) (unknown)
*** Query id: 9464bdf6d2414e4a-87cbdb3036c550cc ***
*** Aborted at 1711621620 (unix time) try "date -d @1711621620" if you are using GNU date ***
*** Current BE git commitID: Unknown ***
*** SIGABRT unkown detail explain (@0x320000140a6) received by PID 82086 (TID 0x7fb1fb684700) from PID 82086; stack trace: ***
I0328 18:27:00.517608 83000 compaction.cpp:400] succeed to do vcumulative compaction is_vertical=0. tablet=378105.1046047283.e14466d63fce2e42-b00dbddb972827bc, output_version=[16350-16555], current_max_version=16555, disk=/data4, segments=6, input_row_num=524111, output_row_num=524111. elapsed time=0.968778s. cumulative_compaction_policy=SIZE_BASED, compact_row_per_second=540997
I0328 18:27:00.651041 83336 tablets_channel.cpp:66] open tablets channel: (id=a343b438331fa9ea-e494903e44270599,index_id=168409), tablets num: 228, timeout(s): 259200
I0328 18:27:00.748097 83389 stream_load.cpp:239] new income streaming load request.id=1a489ddca657ef51-7099fdc1dc8d3eb3, job_id=-1, txn_id=-1, label=flink_connector_20240328_182700_91c06ea2be514549b39f79c8d63fcddb, elapse(s)=0, db=buried_point, tbl=fact_buried_point_hivebox_app_d
I0328 18:27:00.760752 83389 stream_load_executor.cpp:45] begin to execute job. label=flink_connector_20240328_182700_91c06ea2be514549b39f79c8d63fcddb, txn_id=79016619, query_id=1a489ddca657ef51-7099fdc1dc8d3eb3
I0328 18:27:00.760944 83389 fragment_mgr.cpp:642] query_id: 1a489ddca657ef51-7099fdc1dc8d3eb3 coord_addr TNetworkAddress(hostname=xxx, port=9020) total fragment num on current host: 0
I0328 18:27:00.760973 83389 fragment_mgr.cpp:692] Register query/load memory tracker, query/load id: 1a489ddca657ef51-7099fdc1dc8d3eb3 limit: 2.00 GB
I0328 18:27:00.761009 83389 plan_fragment_executor.cpp:92] PlanFragmentExecutor::prepare|query_id=1a489ddca657ef51-7099fdc1dc8d3eb3|instance_id=1a489ddca657ef51-7099fdc1dc8d3eb4|backend_num=0|pthread_id=140394323367680
I0328 18:27:00.769860 5340 fragment_mgr.cpp:503] PlanFragmentExecutor::_exec_actual|query_id=1a489ddca657ef51-7099fdc1dc8d3eb3|instance_id=1a489ddca657ef51-7099fdc1dc8d3eb4|pthread_id=140140269786880
I0328 18:27:00.769913 5340 plan_fragment_executor.cpp:240] PlanFragmentExecutor::open|query_id=1a489ddca657ef51-7099fdc1dc8d3eb3|instance_id=1a489ddca657ef51-7099fdc1dc8d3eb4|mem_limit=2.00 GB
I0328 18:27:00.774906 83315 tablets_channel.cpp:66] open tablets channel: (id=1a489ddca657ef51-7099fdc1dc8d3eb3,index_id=70497), tablets num: 1100, timeout(s): 259200
I0328 18:27:00.807911 82990 compaction.cpp:320] start vcumulative compaction. tablet=377711.628939072.f14d2cb48a5926ac-cf9a3b3bcb2b079f, output_version=[13420-13430], permits: 5
I0328 18:27:00.831861 83319 tablets_channel.cpp:66] open tablets channel: (id=504e32095c4e8539-4eb03b37086e04bb,index_id=132327), tablets num: 83, timeout(s): 259200
I0328 18:27:00.835481 83245 tablets_channel.cpp:102] close tablets channel: (id=504e32095c4e8539-4eb03b37086e04bb,index_id=132327), sender id: 0, backend id: 10005
I0328 18:27:00.835737 83245 load_channel.cpp:44] load channel removed. mem peak usage=0, info=label: LoadChannel#senderIp=xxx#loadID=504e32095c4e8539-4eb03b37086e04bb; consumption: 0; peak_consumption: 0; , load_id=504e32095c4e8539-4eb03b37086e04bb, is high priority=0, sender_ip=xxx, is_vec=1
I0328 18:27:00.841617 82989 compaction.cpp:320] start vcumulative compaction. tablet=377735.628939072.ee4f8ffea2a3da73-2ab8f39d50f909a2, output_version=[13420-13430], permits: 5
I0328 18:27:00.869710 82990 compaction.cpp:400] succeed to do vcumulative compaction is_vertical=0. tablet=377711.628939072.f14d2cb48a5926ac-cf9a3b3bcb2b079f, output_version=[13420-13430], current_max_version=13430, disk=/data2, segments=5, input_row_num=4621, output_row_num=4621. elapsed time=0.061844s. cumulative_compaction_policy=SIZE_BASED, compact_row_per_second=74703
I0328 18:27:00.876912 82990 compaction.cpp:320] start vcumulative compaction. tablet=377715.628939072.8a4616b665cc5453-77daf8990726b89f, output_version=[13400-13430], permits: 5
I0328 18:27:00.910800 82989 compaction.cpp:400] succeed to do vcumulative compaction is_vertical=0. tablet=377735.628939072.ee4f8ffea2a3da73-2ab8f39d50f909a2, output_version=[13420-13430], current_max_version=13430, disk=/data1, segments=5, input_row_num=6589, output_row_num=6589. elapsed time=0.069222s. cumulative_compaction_policy=SIZE_BASED, compact_row_per_second=95170
I0328 18:27:00.913908 82990 compaction.cpp:400] succeed to do vcumulative compaction is_vertical=0. tablet=377715.628939072.8a4616b665cc5453-77daf8990726b89f, output_version=[13400-13430], current_max_version=13430, disk=/data2, segments=5, input_row_num=3012, output_row_num=3012. elapsed time=0.037028s. cumulative_compaction_policy=SIZE_BASED, compact_row_per_second=81321
I0328 18:27:00.916929 82990 compaction.cpp:320] start vcumulative compaction. tablet=377755.628939072.1a4f2a5ddc204879-9902e96350a569ab, output_version=[13420-13430], permits: 5
I0328 18:27:01.009668 5340 vtablet_sink.cpp:421] VNodeChannel[70497-10007], load_id=1a489ddca657ef51-7099fdc1dc8d3eb3, txn_id=79016619, node=xxx:8060 mark closed, left pending batch size: 2
I0328 18:27:01.009718 5340 vtablet_sink.cpp:421] VNodeChannel[70497-10003], load_id=1a489ddca657ef51-7099fdc1dc8d3eb3, txn_id=79016619, node=xxx:8060 mark closed, left pending batch size: 1
I0328 18:27:01.009742 5340 vtablet_sink.cpp:421] VNodeChannel[70497-10006], load_id=1a489ddca657ef51-7099fdc1dc8d3eb3, txn_id=79016619, node=xxx:8060 mark closed, left pending batch size: 1
I0328 18:27:01.009757 5340 vtablet_sink.cpp:421] VNodeChannel[70497-10005], load_id=1a489ddca657ef51-7099fdc1dc8d3eb3, txn_id=79016619, node=xxx:8060 mark closed, left pending batch size: 2
I0328 18:27:01.009761 5340 vtablet_sink.cpp:421] VNodeChannel[70497-10004], load_id=1a489ddca657ef51-7099fdc1dc8d3eb3, txn_id=79016619, node=xxx:8060 mark closed, left pending batch size: 2
I0328 18:27:01.028630 82990 compaction.cpp:400] succeed to do vcumulative compaction is_vertical=0. tablet=377755.628939072.1a4f2a5ddc204879-9902e96350a569ab, output_version=[13420-13430], current_max_version=13430, disk=/data2, segments=5, input_row_num=25157, output_row_num=25157. elapsed time=0.111738s. cumulative_compaction_policy=SIZE_BASED, compact_row_per_second=225120
I0328 18:27:01.041818 83331 tablets_channel.cpp:66] open tablets channel: (id=2349f45298e80ad6-9fc307f5e55b9194,index_id=142111), tablets num: 860, timeout(s): 259200
I0328 18:27:01.047333 83295 tablets_channel.cpp:102] close tablets channel: (id=a343b438331fa9ea-e494903e44270599,index_id=168409), sender id: 0, backend id: 10005
I0328 18:27:01.075428 83370 tablet_sink.cpp:1489] all node channels are stopped(maybe finished/offending/cancelled), sender thread exit. 1a489ddca657ef51-7099fdc1dc8d3eb3
I0328 18:27:01.089497 83129 tablets_channel.cpp:102] close tablets channel: (id=1a489ddca657ef51-7099fdc1dc8d3eb3,index_id=70497), sender id: 0, backend id: 10005
I0328 18:27:01.105520 83295 load_channel.cpp:44] load channel removed. mem peak usage=62030310, info=label: LoadChannel#senderIp=xxx#loadID=a343b438331fa9ea-e494903e44270599; consumption: 62030310; peak_consumption: 62030310; , load_id=a343b438331fa9ea-e494903e44270599, is high priority=0, sender_ip=xxx, is_vec=1
I0328 18:27:01.113570 131071 task_worker_pool.cpp:249] successfully submit task|type=PUBLISH_VERSION|signature=79016618|queue_size=1
I0328 18:27:01.113849 83052 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=168408, tablet=377930.1763621327.8940879c8028549a-bfc7ab240f5c269e, transaction_id=79016618, version=11615, num_rows=9078, res=[OK]
I0328 18:27:01.113855 83060 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=168408, tablet=377926.1763621327.094a6214bffc3766-efb9cad886ad51a7, transaction_id=79016618, version=11615, num_rows=11596, res=[OK]
I0328 18:27:01.113881 83055 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=168408, tablet=376043.1763621327.c24f91fe0ce78343-9467e66522a73da6, transaction_id=79016618, version=26337, num_rows=9, res=[OK]
I0328 18:27:01.113898 83056 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=168408, tablet=376989.1763621327.ac4c5c329ab9b67d-0760cc523d0950b3, transaction_id=79016618, version=20842, num_rows=2, res=[OK]
I0328 18:27:01.113900 83039 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=168408, tablet=376985.1763621327.d64e5bb20627705b-aa7492e4785796bb, transaction_id=79016618, version=20842, num_rows=6, res=[OK]
I0328 18:27:01.113938 83065 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=168408, tablet=376047.1763621327.d24da6722225bfc8-a24d466722078eb9, transaction_id=79016618, version=26337, num_rows=0, res=[OK]
I0328 18:27:01.114015 83084 engine_publish_version_task.cpp:190] finish to publish version on transaction.transaction_id=79016618, cost(us): 435, error_tablet_size=0, res=[OK]
I0328 18:27:01.114045 83084 task_worker_pool.cpp:727] successfully publish version|signature=79016618|transaction_id=79016618|tablets_num=6
I0328 18:27:01.138199 83129 load_channel.cpp:44] load channel removed. mem peak usage=12307271, info=label: LoadChannel#senderIp=xxx#loadID=1a489ddca657ef51-7099fdc1dc8d3eb3; consumption: 12307271; peak_consumption: 12307271; , load_id=1a489ddca657ef51-7099fdc1dc8d3eb3, is high priority=0, sender_ip=xxx, is_vec=1
I0328 18:27:01.139137 5340 tablet_sink.cpp:1207] total mem_exceeded_block_ns=0, total queue_push_lock_ns=135161, total actual_consume_ns=92204070, load id=1a489ddca657ef51-7099fdc1dc8d3eb3
I0328 18:27:01.139194 5340 tablet_sink.cpp:1241] finished to close olap table sink. load_id=1a489ddca657ef51-7099fdc1dc8d3eb3, txn_id=79016619, node add batch time(ms)/wait execution time(ms)/close time(ms)/num: {10004:(92)(0)(130)(3)} {10005:(105)(0)(130)(2)} {10006:(77)(0)(88)(2)} {10003:(82)(0)(84)(2)} {10007:(55)(0)(82)(2)}
I0328 18:27:01.142486 5340 plan_fragment_executor.cpp:715] Close() fragment_instance_id=1a489ddca657ef51-7099fdc1dc8d3eb4
I0328 18:27:01.144760 131071 task_worker_pool.cpp:249] successfully submit task|type=PUBLISH_VERSION|signature=79016619|queue_size=1
I0328 18:27:01.144944 83065 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=70496, tablet=378085.1046047283.98420b9e49a499fd-b67027009acd1dad, transaction_id=79016619, version=16556, num_rows=0, res=[OK]
I0328 18:27:01.145021 83039 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=70496, tablet=378089.1046047283.08401b0ccb7fb856-93df338b33229480, transaction_id=79016619, version=16556, num_rows=8888, res=[OK]
I0328 18:27:01.145042 83055 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=70496, tablet=378105.1046047283.e14466d63fce2e42-b00dbddb972827bc, transaction_id=79016619, version=16556, num_rows=8128, res=[OK]
I0328 18:27:01.145079 83056 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=70496, tablet=378101.1046047283.3a439ba826454094-9d6eea809bb886a5, transaction_id=79016619, version=16556, num_rows=0, res=[OK]
I0328 18:27:01.145126 83085 engine_publish_version_task.cpp:190] finish to publish version on transaction.transaction_id=79016619, cost(us): 358, error_tablet_size=0, res=[OK]
I0328 18:27:01.145154 83085 task_worker_pool.cpp:727] successfully publish version|signature=79016619|transaction_id=79016619|tablets_num=4
I0328 18:27:01.145401 5340 query_fragments_ctx.h:56] Deregister query/load memory tracker, queryId=1a489ddca657ef51-7099fdc1dc8d3eb3, Limit=2.00 GB, CurrUsed=3.84 MB, PeakUsed=39.48 MB
I0328 18:27:01.155061 83389 stream_load.cpp:634] put stream_load_record rocksdb successfully. label: flink_connector_20240328_182700_91c06ea2be514549b39f79c8d63fcddb, key: 1711621621154_flink_connector_20240328_182700_91c06ea2be514549b39f79c8d63fcddb
I0328 18:27:01.248790 82992 compaction.cpp:320] start vcumulative compaction. tablet=376047.1763621327.d24da6722225bfc8-a24d466722078eb9, output_version=[26333-26337], permits: 5
I0328 18:27:01.266014 82992 compaction.cpp:400] succeed to do vcumulative compaction is_vertical=0. tablet=376047.1763621327.d24da6722225bfc8-a24d466722078eb9, output_version=[26333-26337], current_max_version=26337, disk=/data3, segments=3, input_row_num=13, output_row_num=13. elapsed time=0.017263s. cumulative_compaction_policy=SIZE_BASED, compact_row_per_second=752
I0328 18:27:01.383607 83225 tablets_channel.cpp:102] close tablets channel: (id=2349f45298e80ad6-9fc307f5e55b9194,index_id=142111), sender id: 0, backend id: 10005
I0328 18:27:01.402050 82174 daemon.cpp:233] OS physical memory 187.41 GB. Process memory usage 48.70 GB, limit 149.93 GB, soft limit 134.94 GB. Sys available memory 127.67 GB, low water mark 1.60 GB, warning water mark 3.20 GB. Refresh interval memory growth 0 B
I0328 18:27:01.414072 83225 load_channel.cpp:44] load channel removed. mem peak usage=113723064, info=label: LoadChannel#senderIp=xxx#loadID=2349f45298e80ad6-9fc307f5e55b9194; consumption: 113723064; peak_consumption: 113723064; , load_id=2349f45298e80ad6-9fc307f5e55b9194, is high priority=0, sender_ip=xxx, is_vec=1
I0328 18:27:01.437165 131071 task_worker_pool.cpp:249] successfully submit task|type=PUBLISH_VERSION|signature=79016621|queue_size=1
I0328 18:27:01.437464 83044 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=59695, tablet=377707.628939072.974337ae48f00755-4fd4e74c086cc4a3, transaction_id=79016621, version=13431, num_rows=20, res=[OK]
I0328 18:27:01.437475 83035 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=59695, tablet=377719.628939072.fa4aa9e6fe90b56b-8e481b9a8153ec9a, transaction_id=79016621, version=13431, num_rows=2444, res=[OK]
I0328 18:27:01.437515 83042 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=59695, tablet=377711.628939072.f14d2cb48a5926ac-cf9a3b3bcb2b079f, transaction_id=79016621, version=13431, num_rows=431, res=[OK]
I0328 18:27:01.437523 83043 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=59695, tablet=377727.628939072.4643fc7ba9c488af-c089e983a73f33a8, transaction_id=79016621, version=13431, num_rows=1, res=[OK]
I0328 18:27:01.437580 83064 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=59695, tablet=377755.628939072.1a4f2a5ddc204879-9902e96350a569ab, transaction_id=79016621, version=13431, num_rows=2415, res=[OK]
I0328 18:27:01.437582 83059 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=59695, tablet=377759.628939072.f84294dbf46e67c8-82256452a3186580, transaction_id=79016621, version=13431, num_rows=271, res=[OK]
I0328 18:27:01.437605 83054 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=59695, tablet=377715.628939072.8a4616b665cc5453-77daf8990726b89f, transaction_id=79016621, version=13431, num_rows=101, res=[OK]
I0328 18:27:01.437605 83066 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=59695, tablet=377731.628939072.ca47d275743e9751-6b1ef09a04d816a2, transaction_id=79016621, version=13431, num_rows=593, res=[OK]
I0328 18:27:01.437605 83063 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=59695, tablet=377747.628939072.6a4e716c5f6a60e8-fac65b3934910fac, transaction_id=79016621, version=13431, num_rows=14, res=[OK]
I0328 18:27:01.437582 83053 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=59695, tablet=377735.628939072.ee4f8ffea2a3da73-2ab8f39d50f909a2, transaction_id=79016621, version=13431, num_rows=614, res=[OK]
I0328 18:27:01.437667 83040 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=59695, tablet=377739.628939072.a84f2e6acaa2e115-10c71a50f6934f87, transaction_id=79016621, version=13431, num_rows=2004, res=[OK]
I0328 18:27:01.437678 83044 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=59695, tablet=377763.628939072.cb41056bade0d678-ff0416fbc32e27a9, transaction_id=79016621, version=13431, num_rows=27, res=[OK]
I0328 18:27:01.437743 83087 engine_publish_version_task.cpp:190] finish to publish version on transaction.transaction_id=79016621, cost(us): 568, error_tablet_size=0, res=[OK]
I0328 18:27:01.437772 83087 task_worker_pool.cpp:718] trigger quick compaction succ, tabletid:377707, publised:13430
I0328 18:27:01.437813 83087 task_worker_pool.cpp:718] trigger quick compaction succ, tabletid:377719, publised:13430
I0328 18:27:01.437829 83087 task_worker_pool.cpp:718] trigger quick compaction succ, tabletid:377711, publised:13430
I0328 18:27:01.437839 83087 task_worker_pool.cpp:718] trigger quick compaction succ, tabletid:377727, publised:13430
I0328 18:27:01.437853 83087 task_worker_pool.cpp:718] trigger quick compaction succ, tabletid:377755, publised:13430
I0328 18:27:01.437878 83087 task_worker_pool.cpp:718] trigger quick compaction succ, tabletid:377759, publised:13430
I0328 18:27:01.437897 83087 task_worker_pool.cpp:718] trigger quick compaction succ, tabletid:377735, publised:13430
I0328 18:27:01.437911 83087 task_worker_pool.cpp:718] trigger quick compaction succ, tabletid:377731, publised:13430
I0328 18:27:01.437923 83087 task_worker_pool.cpp:718] trigger quick compaction succ, tabletid:377715, publised:13430
I0328 18:27:01.437950 83087 task_worker_pool.cpp:718] trigger quick compaction succ, tabletid:377747, publised:13430
I0328 18:27:01.437965 83087 task_worker_pool.cpp:718] trigger quick compaction succ, tabletid:377739, publised:13430
I0328 18:27:01.437978 83087 task_worker_pool.cpp:718] trigger quick compaction succ, tabletid:377763, publised:13430
I0328 18:27:01.437985 83087 task_worker_pool.cpp:727] successfully publish version|signature=79016621|transaction_id=79016621|tablets_num=12
I0328 18:27:01.724102 83113 task_worker_pool.cpp:1493] successfully report TASK|host=xxx|port=9020
I0328 18:27:01.831671 83318 tablets_channel.cpp:66] open tablets channel: (id=694bbc0b22f8e96a-b371b158ce8b1fbd,index_id=70497), tablets num: 1100, timeout(s): 259200
I0328 18:27:02.007294 82174 daemon.cpp:233] OS physical memory 187.41 GB. Process memory usage 48.59 GB, limit 149.93 GB, soft limit 134.94 GB. Sys available memory 127.79 GB, low water mark 1.60 GB, warning water mark 3.20 GB. Refresh interval memory growth 0 B
I0328 18:27:02.017529 83161 tablets_channel.cpp:102] close tablets channel: (id=694bbc0b22f8e96a-b371b158ce8b1fbd,index_id=70497), sender id: 0, backend id: 10005
I0328 18:27:02.051820 83161 load_channel.cpp:44] load channel removed. mem peak usage=8319381, info=label: LoadChannel#senderIp=xxx#loadID=694bbc0b22f8e96a-b371b158ce8b1fbd; consumption: 8319381; peak_consumption: 8319381; , load_id=694bbc0b22f8e96a-b371b158ce8b1fbd, is high priority=0, sender_ip=xxx, is_vec=1
I0328 18:27:02.061805 131071 task_worker_pool.cpp:249] successfully submit task|type=PUBLISH_VERSION|signature=79016622|queue_size=1
I0328 18:27:02.062063 83050 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=70496, tablet=378089.1046047283.08401b0ccb7fb856-93df338b33229480, transaction_id=79016622, version=16557, num_rows=4064, res=[OK]
I0328 18:27:02.062127 83037 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=70496, tablet=378105.1046047283.e14466d63fce2e42-b00dbddb972827bc, transaction_id=79016622, version=16557, num_rows=8128, res=[OK]
I0328 18:27:02.062136 83043 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=70496, tablet=378101.1046047283.3a439ba826454094-9d6eea809bb886a5, transaction_id=79016622, version=16557, num_rows=4064, res=[OK]
I0328 18:27:02.062196 83057 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=70496, tablet=378085.1046047283.98420b9e49a499fd-b67027009acd1dad, transaction_id=79016622, version=16557, num_rows=0, res=[OK]
I0328 18:27:02.062251 83089 engine_publish_version_task.cpp:190] finish to publish version on transaction.transaction_id=79016622, cost(us): 439, error_tablet_size=0, res=[OK]
I0328 18:27:02.062286 83089 task_worker_pool.cpp:727] successfully publish version|signature=79016622|transaction_id=79016622|tablets_num=4
I0328 18:27:02.089130 83333 tablets_channel.cpp:66] open tablets channel: (id=714e528c11b7618f-51dd1dc583470891,index_id=68674), tablets num: 338, timeout(s): 259200
I0328 18:27:02.174700 83291 tablets_channel.cpp:102] close tablets channel: (id=714e528c11b7618f-51dd1dc583470891,index_id=68674), sender id: 0, backend id: 10005
I0328 18:27:02.186883 83291 load_channel.cpp:44] load channel removed. mem peak usage=0, info=label: LoadChannel#senderIp=xx#loadID=714e528c11b7618f-51dd1dc583470891; consumption: 0; peak_consumption: 0; , load_id=714e528c11b7618f-51dd1dc583470891, is high priority=0, sender_ip=xxx, is_vec=1
I0328 18:27:02.193037 131071 task_worker_pool.cpp:249] successfully submit task|type=PUBLISH_VERSION|signature=79016623|queue_size=1
I0328 18:27:02.193230 83055 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=68673, tablet=379679.1145043063.bc472ac9cb17060d-b400f61d5ee27a91, transaction_id=79016623, version=8836, num_rows=4064, res=[OK]
I0328 18:27:02.193243 83056 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=68673, tablet=379687.1145043063.7f405c3042045464-a56e1fc108b3c480, transaction_id=79016623, version=8836, num_rows=0, res=[OK]
I0328 18:27:02.193279 83039 engine_publish_version_task.cpp:237] publish version successfully on tablet, table_id=68673, tablet=379683.1145043063.da4c8cf006e9e1d4-e9bb018d3ae87e96, transaction_id=79016623, version=8836, num_rows=4706, res=[OK]
I0328 18:27:02.193320 83082 engine_publish_version_task.cpp:190] finish to publish version on transaction.transaction_id=79016623, cost(us): 277, error_tablet_size=0, res=[OK]
I0328 18:27:02.193349 83082 task_worker_pool.cpp:727] successfully publish version|signature=79016623|transaction_id=79016623|tablets_num=3
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /root/doris/be/src/common/signal_handler.h:420
1# 0x00007FB270A3C400 in /lib64/libc.so.6
2# __GI_raise in /lib64/libc.so.6
3# abort in /lib64/libc.so.6
4# 0x00005580F357D699 in /app/apache-doris/be/lib/doris_be
5# 0x00005580F3572CAD in /app/apache-doris/be/lib/doris_be
6# google::LogMessage::SendToLog() in /app/apache-doris/be/lib/doris_be
7# google::LogMessage::Flush() in /app/apache-doris/be/lib/doris_be
8# google::LogMessageFatal::~LogMessageFatal() in /app/apache-doris/be/lib/doris_be
9# doris::stream_load::VNodeChannel::try_send_block(doris::RuntimeState*) at /root/doris/be/src/vec/sink/vtablet_sink.cpp:354
10# doris::ThreadPool::dispatch_thread() at /root/doris/be/src/util/threadpool.cpp:543
11# doris::Thread::supervise_thread(void*) at /root/doris/be/src/util/thread.cpp:455
12# start_thread in /lib64/libpthread.so.0
13# clone in /lib64/libc.so.6
引用
/app/apache-doris/be/bin/start_be.sh: line 323: 82086 Aborted (core dumped) ${LIMIT:+${LIMIT}} "${DORIS_HOME}/lib/doris_be" "$@" 2>&1 < /dev/null

be.out里面只有如下日志

start time: Wed Mar 6 21:47:57 CST 2024
start time: Thu Mar 28 18:34:53 CST 2024

1 Answers

去fe.audit.log里找一下这个对应query语句执行一下,看看能否稳定复现
image.png