集群信息:3台be,3台fe 稳定运行1年多。
异常信息:升级到 1.2.6后出现2次 be异常退出
be.out 信息如下
*** Query id: 405e89dcf9b40c4-aa38736608494dfe ***
*** Aborted at 1711590265 (unix time) try "date -d @1711590265" if you are using GNU date ***
*** Current BE git commitID: Unknown ***
*** SIGSEGV invalid permissions for mapped object (@0x7fb41db68000) received by PID 288359 (TID 0x7f9b27083700) from PID 498499584; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /root/doris/be/src/common/signal_handler.h:420
1# os::Linux::chained_handler(int, siginfo*, void*) in /usr/local/jdk_current/jre/lib/amd64/server/libjvm.so
2# JVM_handle_linux_signal in /usr/local/jdk_current/jre/lib/amd64/server/libjvm.so
3# signalHandler(int, siginfo*, void*) in /usr/local/jdk_current/jre/lib/amd64/server/libjvm.so
4# 0x00007FB6FBFD5400 in /lib64/libc.so.6
5# _ZNSt8__detail9__variant17__gen_vtable_implINS0_12_Multi_arrayIPFNS0_21__deduce_visit_resultIN5doris6StatusEEEONS4_10vectorized8OverloadIJZNS7_12HashJoinNode20_process_build_blockEPNS4_12RuntimeStateERNS7_5BlockEhEUlRSt9monostateT_T0_E_ZNS9_20_process_build_blockESB_SD_hEUlOSG_SH_T1_E0_EEERSt7variantIJSE_NS7_26SerializedHashTableContextINS7_10RowRefListEEENS7_27PrimaryTypeHashTableContextIhSQ_EENSS_ItSQ_EENSS_IjSQ_EENSS_ImSQ_EENSS_INS7_7UInt128ESQ_EENSS_INS7_7UInt256ESQ_EENS7_24FixedKeyHashTableContextImLb1ESQ_EENS11_ImLb0ESQ_EENS11_ISX_Lb1ESQ_EENS11_ISX_Lb0ESQ_EENS11_ISZ_Lb1ESQ_EENS11_ISZ_Lb0ESQ_EENSP_INS7_18RowRefListWithFlagEEENSS_IhS18_EENSS_ItS18_EENSS_IjS18_EENSS_ImS18_EENSS_ISX_S18_EENSS_ISZ_S18_EENS11_ImLb1ES18_EENS11_ImLb0ES18_EENS11_ISX_Lb1ES18_EENS11_ISX_Lb0ES18_EENS11_ISZ_Lb1ES18_EENS11_ISZ_Lb0ES18_EENSP_INS7_19RowRefListWithFlagsEEENSS_IhS1M_EENSS_ItS1M_EENSS_IjS1M_EENSS_ImS1M_EENSS_ISX_S1M_EENSS_ISZ_S1M_EENS11_ImLb1ES1M_EENS11_ImLb0ES1M_EENS11_ISX_Lb1ES1M_EENS11_ISX_Lb0ES1M_EENS11_ISZ_Lb1ES1M_EENS11_ISZ_Lb0ES1M_EEEEOSO_IJSt17integral_constantIbLb0EES22_IbLb1EEEES26_EJEEESt16integer_sequenceImJLm9ELm0ELm0EEEE14__visit_invokeESN_S21_S26_S26_ at /var/local/ldb-toolchain/include/c++/11/variant:1015
6# doris::vectorized::HashJoinNode::_process_build_block(doris::RuntimeState*, doris::vectorized::Block&, unsigned char) at /root/doris/be/src/vec/exec/join/vhash_join_node.cpp:933
7# doris::vectorized::HashJoinNode::_materialize_build_side(doris::RuntimeState*) at /root/doris/be/src/vec/exec/join/vhash_join_node.cpp:710
8# doris::vectorized::VJoinNodeBase::open(doris::RuntimeState*) at /root/doris/be/src/vec/exec/join/vjoin_node_base.cpp:203
9# doris::vectorized::HashJoinNode::open(doris::RuntimeState*) at /root/doris/be/src/vec/exec/join/vhash_join_node.cpp:650
10# doris::vectorized::AggregationNode::open(doris::RuntimeState*) at /root/doris/be/src/vec/exec/vaggregation_node.cpp:458
11# doris::PlanFragmentExecutor::open_vectorized_internal() at /root/doris/be/src/runtime/plan_fragment_executor.cpp:289
12# doris::PlanFragmentExecutor::open() at /root/doris/be/src/runtime/plan_fragment_executor.cpp:261
13# doris::FragmentExecState::execute() at /root/doris/be/src/runtime/fragment_mgr.cpp:261
14# doris::FragmentMgr::_exec_actual(std::shared_ptr<doris::FragmentExecState>, std::function<void (doris::PlanFragmentExecutor*)>) at /root/doris/be/src/runtime/fragment_mgr.cpp:508
15# std::_Function_handler<void (), doris::FragmentMgr::exec_plan_fragment(doris::TExecPlanFragmentParams const&, std::function<void (doris::PlanFragmentExecutor*)>)::{lambda()#1}>::_M_invoke(std::_Any_data const&) at /var/local/ldb-toolchain/include/c++/11/bits/std_function.h:291
16# doris::ThreadPool::dispatch_thread() at /root/doris/be/src/util/threadpool.cpp:543
17# doris::Thread::supervise_thread(void*) at /root/doris/be/src/util/thread.cpp:455
18# start_thread in /lib64/libpthread.so.0
19# __clone in /lib64/libc.so.6
通过 Tid 和审计日志,获取到异常时候最后一条sql, 单独提取sql 并执行,可以正常返回结果