【已解决】单查询不报错,插入报错[E-3113][E-3113] string column length is too large: total_length=4295003210, element_number=429500322

Viewed 204

将查询的结果插入到外表时报错,但是只查询不插入正常。报错内容如下:

看报错内容,malloc or new large memory: 4294967296,不知道是不是有个参数可以设置,还是一次最多分配4G?

W20240417 18:27:04.531735 205747 thread_mem_tracker_mgr.h:184] malloc or new large memory: 4294967296, in query or load: 6de086fdcbd84ac1-a591a43e5f53c0da, this is just a warning, not prevent memory alloc, stacktrace:

	0#  doris::ThreadMemTrackerMgr::consume(long, int) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187
	1#  Allocator<false, false, false>::realloc_impl(void*, unsigned long, unsigned long, unsigned long) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/common/allocator.h:173
	2#  doris::vectorized::ColumnVector<__int128>::insert_range_from(doris::vectorized::IColumn const&, unsigned long, unsigned long) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/common/pod_array.h:171
	3#  doris::vectorized::ColumnNullable::insert_range_from(doris::vectorized::IColumn const&, unsigned long, unsigned long) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/common/cow.h:194
	4#  doris::Status doris::vectorized::MutableBlock::merge_impl<doris::vectorized::Block&>(doris::vectorized::Block&) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/common/cow.h:132
	5#  doris::Status doris::vectorized::MutableBlock::merge<doris::vectorized::Block&>(doris::vectorized::Block&) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/core/block.h:512
	6#  doris::pipeline::HashJoinBuildSinkOperatorX::sink(doris::RuntimeState*, doris::vectorized::Block*, bool) at /home/zcp/repo_center/doris_enterprise/doris/be/src/pipeline/exec/hashjoin_build_sink.cpp:0
	7#  doris::pipeline::PipelineXTask::execute(bool*) at /home/zcp/repo_center/doris_enterprise/doris/be/src/common/status.h:345
	8#  doris::pipeline::TaskScheduler::_do_work(unsigned long) at /home/zcp/repo_center/doris_enterprise/doris/be/src/common/status.h:345
	9#  doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_enterprise/doris/be/src/util/threadpool.cpp:0
	10# doris::Thread::supervise_thread(void*) at /var/local/ldb_toolchain/bin/../usr/include/pthread.h:562
	11# start_thread
	12# clone





 Pipeline task failed. query_id: 0-0|6de086fdcbd84ac1-a591a43e5f53c0da reason: [E-3113][E-3113] string column length is too large: total_length=4295003210, element_number=429500322

	0#  doris::Exception::Exception(int, std::basic_string_view<char, std::char_traits<char> > const&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:173
	1#  doris::Exception::Exception<unsigned long&, unsigned long&>(int, std::basic_string_view<char, std::char_traits<char> > const&, unsigned long&, unsigned long&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187
	2#  doris::vectorized::ColumnString::insert_range_from(doris::vectorized::IColumn const&, unsigned long, unsigned long) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/columns/column_string.cpp:0
	3#  doris::vectorized::ColumnNullable::insert_range_from(doris::vectorized::IColumn const&, unsigned long, unsigned long) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/common/cow.h:194
	4#  doris::Status doris::vectorized::MutableBlock::merge_impl<doris::vectorized::Block&>(doris::vectorized::Block&) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/common/cow.h:132
	5#  doris::Status doris::vectorized::MutableBlock::merge<doris::vectorized::Block&>(doris::vectorized::Block&) at /home/zcp/repo_center/doris_enterprise/doris/be/src/vec/core/block.h:512
	6#  doris::pipeline::HashJoinBuildSinkOperatorX::sink(doris::RuntimeState*, doris::vectorized::Block*, bool) at /home/zcp/repo_center/doris_enterprise/doris/be/src/pipeline/exec/hashjoin_build_sink.cpp:0
	7#  doris::pipeline::PipelineXTask::execute(bool*) at /home/zcp/repo_center/doris_enterprise/doris/be/src/common/status.h:345
	8#  doris::pipeline::TaskScheduler::_do_work(unsigned long) at /home/zcp/repo_center/doris_enterprise/doris/be/src/common/status.h:345
	9#  doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_enterprise/doris/be/src/util/threadpool.cpp:0
	10# doris::Thread::supervise_thread(void*) at /var/local/ldb_toolchain/bin/../usr/include/pthread.h:562
	11# start_thread
	12# clone
2 Answers

2.1.3 会彻底fix 这个问题