Doris 2.1.3 BE启动后每24小时会自己停止

Question

Doris 2.1.3版本，3个FE, 3个BE，FE可以连续稳定运行，但三个BE节点每运行24小时就会出现宕机，开始认为是系统连接限制，后来改为system service，但还是24小时出现宕机的问题。下面是2次出现问题后的be.out文件的输出：

start time: Tue Jun 25 23:04:38 CST 2024
INFO: java_cmd /data1/jdk1.8.0_202//bin/java
INFO: jdk_version 8
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/data1/doris/be/lib/java_extensions/preload-extensions/preload-extensions-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/data1/doris/be/lib/java_extensions/java-udf/java-udf-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/data1/doris/be/lib/hadoop_hdfs/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
*** Query id: 0-0 ***
*** is nereids: 0 ***
*** tablet id: 10356 ***
*** Aborted at 1719414448 (unix time) try "date -d @1719414448" if you are using GNU date ***
*** Current BE git commitID: 2dc65ce356 ***
*** SIGSEGV address not mapped to object (@0x0) received by PID 17371 (TID 18521 OR 0x7f114dfd9700) from PID 0; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_release/doris/be/src/common/signal_handler.h:421
1# os::Linux::chained_handler(int, siginfo*, void*) in /data1/jdk1.8.0_202/jre/lib/amd64/server/libjvm.so
2# JVM_handle_linux_signal in /data1/jdk1.8.0_202/jre/lib/amd64/server/libjvm.so
3# signalHandler(int, siginfo*, void*) in /data1/jdk1.8.0_202/jre/lib/amd64/server/libjvm.so
4# 0x00007F13CD4F4280 in /lib64/libc.so.6
5# doris::segment_v2::SegmentWriter::_full_encode_keys[abi:cxx11](std::vector > const&, std::vector const&, unsigned long, bool) at /home/zcp/repo_center/doris_release/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:913
6# doris::segment_v2::SegmentWriter::_generate_short_key_index(std::vector&, unsigned long, std::vector const&) at /home/zcp/repo_center/doris_release/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:1297
7# doris::segment_v2::SegmentWriter::append_block(doris::vectorized::Block const*, unsigned long, unsigned long) in /data1/doris/be/lib/doris_be
8# doris::VerticalBetaRowsetWriter::add_columns(doris::vectorized::Block const*, std::vector const&, bool, unsigned int) at /home/zcp/repo_center/doris_release/doris/be/src/olap/rowset/vertical_beta_rowset_writer.cpp:85
9# doris::Merger::vertical_compact_one_group(std::shared_ptr, doris::ReaderType, std::shared_ptr, bool, std::vector const&, doris::vectorized::RowSourcesBuffer*, std::vector > const&, doris::RowsetWriter*, long, doris::Merger::Statistics*, std::vector) in /data1/doris/be/lib/doris_be
10# doris::Merger::vertical_merge_rowsets(std::shared_ptr, doris::ReaderType, std::shared_ptr, std::vector > const&, doris::RowsetWriter*, long, doris::Merger::Statistics*) at /home/zcp/repo_center/doris_release/doris/be/src/olap/merger.cpp:383
11# doris::Compaction::do_compaction_impl(long) at /home/zcp/repo_center/doris_release/doris/be/src/olap/compaction.cpp:371
12# doris::Compaction::do_compaction(long) at /home/zcp/repo_center/doris_release/doris/be/src/olap/compaction.cpp:136
13# doris::BaseCompaction::execute_compact_impl() at /home/zcp/repo_center/doris_release/doris/be/src/olap/base_compaction.cpp:79
14# doris::Compaction::execute_compact() at /home/zcp/repo_center/doris_release/doris/be/src/olap/compaction.cpp:118
15# doris::Tablet::execute_compaction(doris::Compaction&) at /home/zcp/repo_center/doris_release/doris/be/src/olap/tablet.cpp:1947
16# std::_Function_handler::_M_invoke(std::_Any_data const&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
17# doris::ThreadPool::dispatch_thread() in /data1/doris/be/lib/doris_be
18# doris::Thread::supervise_thread(void*) at /home/zcp/repo_center/doris_release/doris/be/src/util/thread.cpp:499
19# start_thread in /lib64/libpthread.so.0
20# __clone in /lib64/libc.so.6

start time: Fri Jun 28 22:05:59 CST 2024
INFO: java_cmd /data1/jdk1.8.0_202//bin/java
INFO: jdk_version 8
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/data1/doris/be/lib/java_extensions/preload-extensions/preload-extensions-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/data1/doris/be/lib/java_extensions/java-udf/java-udf-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/data1/doris/be/lib/hadoop_hdfs/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
*** Query id: 0-0 ***
*** is nereids: 0 ***
*** tablet id: 10356 ***
*** Aborted at 1719670546 (unix time) try "date -d @1719670546" if you are using GNU date ***
*** Current BE git commitID: 2dc65ce356 ***
*** SIGSEGV address not mapped to object (@0x0) received by PID 10017 (TID 11345 OR 0x7f112f771700) from PID 0; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_release/doris/be/src/common/signal_handler.h:421
1# os::Linux::chained_handler(int, siginfo*, void*) in /data1/jdk1.8.0_202/jre/lib/amd64/server/libjvm.so
2# JVM_handle_linux_signal in /data1/jdk1.8.0_202/jre/lib/amd64/server/libjvm.so
3# signalHandler(int, siginfo*, void*) in /data1/jdk1.8.0_202/jre/lib/amd64/server/libjvm.so
4# 0x00007F13A78CF280 in /lib64/libc.so.6
5# doris::segment_v2::SegmentWriter::_full_encode_keys[abi:cxx11](std::vector > const&, std::vector const&, unsigned long, bool) at /home/zcp/repo_center/doris_release/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:913
6# doris::segment_v2::SegmentWriter::_generate_short_key_index(std::vector&, unsigned long, std::vector const&) at /home/zcp/repo_center/doris_release/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:1297
7# doris::segment_v2::SegmentWriter::append_block(doris::vectorized::Block const*, unsigned long, unsigned long) in /data1/doris/be/lib/doris_be
8# doris::VerticalBetaRowsetWriter::add_columns(doris::vectorized::Block const*, std::vector const&, bool, unsigned int) at /home/zcp/repo_center/doris_release/doris/be/src/olap/rowset/vertical_beta_rowset_writer.cpp:85
9# doris::Merger::vertical_compact_one_group(std::shared_ptr, doris::ReaderType, std::shared_ptr, bool, std::vector const&, doris::vectorized::RowSourcesBuffer*, std::vector > const&, doris::RowsetWriter*, long, doris::Merger::Statistics*, std::vector) in /data1/doris/be/lib/doris_be
10# doris::Merger::vertical_merge_rowsets(std::shared_ptr, doris::ReaderType, std::shared_ptr, std::vector > const&, doris::RowsetWriter*, long, doris::Merger::Statistics*) at /home/zcp/repo_center/doris_release/doris/be/src/olap/merger.cpp:383
11# doris::Compaction::do_compaction_impl(long) at /home/zcp/repo_center/doris_release/doris/be/src/olap/compaction.cpp:371
12# doris::Compaction::do_compaction(long) at /home/zcp/repo_center/doris_release/doris/be/src/olap/compaction.cpp:136
13# doris::BaseCompaction::execute_compact_impl() at /home/zcp/repo_center/doris_release/doris/be/src/olap/base_compaction.cpp:79
14# doris::Compaction::execute_compact() at /home/zcp/repo_center/doris_release/doris/be/src/olap/compaction.cpp:118
15# doris::Tablet::execute_compaction(doris::Compaction&) at /home/zcp/repo_center/doris_release/doris/be/src/olap/tablet.cpp:1947
16# std::_Function_handler::_M_invoke(std::_Any_data const&) at /var/local/ldb_toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291
17# doris::ThreadPool::dispatch_thread() in /data1/doris/be/lib/doris_be
18# doris::Thread::supervise_thread(void*) at /home/zcp/repo_center/doris_release/doris/be/src/util/thread.cpp:499
19# start_thread in /lib64/libpthread.so.0
20# __clone in /lib64/libc.so.6

请帮助指教，谢谢！

黄海军@SelectDB · Answer

【问题状态】处理中
【问题处理】已记录，有进展会更新回帖，方便的话加我一下微信(hhj_0530)方便后续沟通

Sean · Answer

如下命令执行后的返回结果：

MySQL [testdb]> show tablet 10356 \G;
*************************** 1. row ***************************
DbName: testdb
TableName: doris_test_10
PartitionName: doris_test_10
IndexName: doris_test_10
DbId: 10321
TableId: 10324
PartitionId: 10323
IndexId: 10325
IsSync: true
Order: 10
QueryHits: 0
DetailCmd: SHOW PROC '/dbs/10321/10324/partitions/10323/10325/10356';
1 row in set (0.01 sec)

MySQL [testdb]> SHOW PROC '/dbs/10321/10324/partitions/10323/10325/10356' \G;
*************************** 1. row ***************************
ReplicaId: 10358
BackendId: 10199
Version: 5
LstSuccessVersion: 5
LstFailedVersion: -1
LstFailedTime: NULL
SchemaHash: 420352195
LocalDataSize: 1152
RemoteDataSize: 0
RowCount: 1
State: NORMAL
IsBad: false
IsUserDrop: false
VersionCount: 5
PathHash: -7587543994326539009
Path: /data1/doris/be/storage
MetaUrl: http://10.122.67.113:8040/api/meta/header/10356
CompactionStatus: http://10.122.67.113:8040/api/compaction/show?tablet_id=10356
CooldownReplicaId: -1
CooldownMetaId:
QueryHits: 0
*************************** 2. row ***************************
ReplicaId: 10357
BackendId: 10276
Version: 5
LstSuccessVersion: 5
LstFailedVersion: -1
LstFailedTime: NULL
SchemaHash: 420352195
LocalDataSize: 1152
RemoteDataSize: 0
RowCount: 1
State: NORMAL
IsBad: false
IsUserDrop: false
VersionCount: 5
PathHash: 5080673796539278104
Path: /data1/doris/be/storage
MetaUrl: http://10.122.67.114:8040/api/meta/header/10356
CompactionStatus: http://10.122.67.114:8040/api/compaction/show?tablet_id=10356
CooldownReplicaId: -1
CooldownMetaId:
QueryHits: 0
2 rows in set (0.00 sec)

CompactionStatus返回的结果：
{
"cumulative policy type": "size_based",
"cumulative point": 2,
"last cumulative failure time": "2024-07-07 14:43:49.832",
"last base failure time": "1970-01-01 08:00:00.000",
"last full failure time": "1970-01-01 08:00:00.000",
"last cumulative success time": "2024-07-06 22:49:00.026",
"last base success time": "2024-07-06 22:49:00.026",
"last full success time": "1970-01-01 08:00:00.000",
"last base schedule time": "1970-01-01 08:00:00.000",
"last base status": "",
"rowsets": [
"[0-1] 0 DATA OVERLAP_UNKNOWN 020000000000001e39458223160034a874aead8a36248297 0",
"[2-2] 0 DATA OVERLAPPING 020000000000002439458223160034a874aead8a36248297 0",
"[3-3] 1 DATA NONOVERLAPPING 020000000000002d39458223160034a874aead8a36248297 1.12 KB",
"[4-4] 0 DATA OVERLAPPING 020000000000003b39458223160034a874aead8a36248297 0",
"[5-5] 0 DATA OVERLAPPING 020000000000004639458223160034a874aead8a36248297 0"
],
"missing_rowsets": [],
"stale_rowsets": [],
"stale version path": []
}

Metaurl返回的结果：
{
"table_id": 10324,
"partition_id": 10323,
"tablet_id": 10356,
"schema_hash": 420352195,
"shard_id": 29,
"creation_time": 1718380051,
"cumulative_layer_point": -1,
"tablet_state": "PB_RUNNING",
"schema": {
"keys_type": "DUP_KEYS",
"column": [
{
"unique_id": 0,
"name": "message_time",
"type": "DATETIMEV2",
"is_key": true,
"aggregation": "NONE",
"is_nullable": true,
"precision": 18,
"frac": 0,
"length": 8,
"index_length": 8,
"visible": true,
"result_is_nullable": false
},
{
"unique_id": 1,
"name": "message_body",
"type": "VARIANT",
"is_key": true,
"aggregation": "NONE",
"is_nullable": true,
"precision": 0,
"frac": 0,
"length": 65537,
"index_length": 65537,
"visible": true,
"result_is_nullable": false
}
],
"num_short_key_columns": 2,
"num_rows_per_row_block": 1024,
"compress_kind": "COMPRESS_LZ4",
"next_column_unique_id": 2,
"is_in_memory": false,
"delete_sign_idx": -1,
"sequence_col_idx": -1,
"sort_type": "LEXICAL",
"sort_col_num": 2,
"compression_type": "LZ4F",
"schema_version": 0,
"disable_auto_compaction": false,
"index": [
{
"index_id": 10322,
"index_name": "idx_var",
"index_type": "INVERTED",
"col_unique_id": [
1
],
"index_suffix_name": "",
"properties": {
"lower_case": "true"
}
}
],
"version_col_idx": -1,
"store_row_column": false,
"enable_single_replica_compaction": false,
"skip_write_index_on_load": false,
"inverted_index_storage_format": "V1"
},
"rs_metas": [
{
"rowset_id": 0,
"partition_id": 10323,
"tablet_id": 10356,
"tablet_schema_hash": 420352195,
"rowset_type": "BETA_ROWSET",
"rowset_state": "VISIBLE",
"start_version": 0,
"end_version": 1,
"num_rows": 0,
"total_disk_size": 0,
"data_disk_size": 0,
"index_disk_size": 0,
"empty": true,
"creation_time": 1718380051,
"tablet_uid": {
"hi": 2470781828009500345,
"lo": 6867912887031541946
},
"num_segments": 0,
"rowset_id_v2": "020000000000001e39458223160034a874aead8a36248297",
"newest_write_timestamp": 1718380051,
"tablet_schema": {
"keys_type": "DUP_KEYS",
"column": [
{
"unique_id": 0,
"name": "message_time",
"type": "DATETIMEV2",
"is_key": true,
"aggregation": "NONE",
"is_nullable": true,
"precision": 18,
"frac": 0,
"length": 8,
"index_length": 8,
"visible": true,
"result_is_nullable": false
},
{
"unique_id": 1,
"name": "message_body",
"type": "VARIANT",
"is_key": true,
"aggregation": "NONE",
"is_nullable": true,
"precision": 0,
"frac": 0,
"length": 65537,
"index_length": 65537,
"visible": true,
"result_is_nullable": false
}
],
"num_short_key_columns": 2,
"num_rows_per_row_block": 1024,
"compress_kind": "COMPRESS_LZ4",
"next_column_unique_id": 2,
"is_in_memory": false,
"delete_sign_idx": -1,
"sequence_col_idx": -1,
"sort_type": "LEXICAL",
"sort_col_num": 2,
"compression_type": "LZ4F",
"schema_version": 0,
"disable_auto_compaction": false,
"index": [
{
"index_id": 10322,
"index_name": "idx_var",
"index_type": "INVERTED",
"col_unique_id": [
1
],
"index_suffix_name": "",
"properties": {
"lower_case": "true"
}
}
],
"version_col_idx": -1,
"store_row_column": false,
"enable_single_replica_compaction": false,
"skip_write_index_on_load": false,
"inverted_index_storage_format": "V1"
},
"segments_overlap_pb": "OVERLAP_UNKNOWN"
},
{
"rowset_id": 0,
"partition_id": 10323,
"tablet_id": 10356,
"txn_id": 2,
"tablet_schema_hash": 420352195,
"rowset_type": "BETA_ROWSET",
"rowset_state": "VISIBLE",
"start_version": 2,
"end_version": 2,
"num_rows": 0,
"total_disk_size": 0,
"data_disk_size": 0,
"index_disk_size": 0,
"empty": true,
"load_id": {
"hi": 9110588469552629772,
"lo": -5575841013962358078
},
"creation_time": 1718380057,
"tablet_uid": {
"hi": 2470781828009500345,
"lo": 6867912887031541946
},
"num_segments": 0,
"rowset_id_v2": "020000000000002439458223160034a874aead8a36248297",
"newest_write_timestamp": 1718380057,
"tablet_schema": {
"keys_type": "DUP_KEYS",
"column": [
{
"unique_id": 0,
"name": "message_time",
"type": "DATETIMEV2",
"is_key": true,
"aggregation": "NONE",
"is_nullable": true,
"precision": 18,
"frac": 0,
"length": 8,
"index_length": 8,
"visible": true,
"result_is_nullable": false
},
{
"unique_id": 1,
"name": "message_body",
"type": "VARIANT",
"is_key": true,
"aggregation": "NONE",
"is_nullable": true,
"precision": 0,
"frac": 0,
"length": 65537,
"index_length": 65537,
"visible": true,
"result_is_nullable": false
}
],
"num_short_key_columns": 2,
"num_rows_per_row_block": 1024,
"compress_kind": "COMPRESS_LZ4",
"next_column_unique_id": 2,
"is_in_memory": false,
"delete_sign_idx": -1,
"sequence_col_idx": -1,
"sort_type": "LEXICAL",
"sort_col_num": 2,
"compression_type": "LZ4F",
"schema_version": 0,
"disable_auto_compaction": false,
"index": [
{
"index_id": 10322,
"index_name": "idx_var",
"index_type": "INVERTED",
"col_unique_id": [
1
],
"index_suffix_name": "",
"properties": {
"lower_case": "true"
}
}
],
"version_col_idx": -1,
"store_row_column": false,
"enable_single_replica_compaction": false,
"skip_write_index_on_load": false,
"inverted_index_storage_format": "V1"
},
"segments_overlap_pb": "OVERLAPPING"
},
{
"rowset_id": 0,
"partition_id": 10323,
"tablet_id": 10356,
"txn_id": 3,
"tablet_schema_hash": 420352195,
"rowset_type": "BETA_ROWSET",
"rowset_state": "VISIBLE",
"start_version": 3,
"end_version": 3,
"num_rows": 1,
"total_disk_size": 1152,
"data_disk_size": 1152,
"index_disk_size": 480,
"empty": false,
"load_id": {
"hi": -185353677791016267,
"lo": -7446249720526495349
},
"creation_time": 1718380060,
"tablet_uid": {
"hi": 2470781828009500345,
"lo": 6867912887031541946
},
"num_segments": 1,
"rowset_id_v2": "020000000000002d39458223160034a874aead8a36248297",
"newest_write_timestamp": 1718380060,
"segments_key_bounds": [
{
"min_key": "\u0002\u0001�\u0019׾�\u0000\u0000\u0001",
"max_key": "\u0002\u0001�\u0019׾�\u0000\u0000\u0001"
}
],
"tablet_schema": {
"keys_type": "DUP_KEYS",
"column": [
{
"unique_id": 0,
"name": "message_time",
"type": "DATETIMEV2",
"is_key": true,
"aggregation": "NONE",
"is_nullable": true,
"precision": 18,
"frac": 0,
"length": 8,
"index_length": 8,
"visible": true,
"result_is_nullable": false
},
{
"unique_id": 1,
"name": "message_body",
"type": "VARIANT",
"is_key": true,
"aggregation": "NONE",
"is_nullable": true,
"precision": 0,
"frac": 0,
"length": 65537,
"index_length": 0,
"visible": true,
"result_is_nullable": false,
"column_path_info": {
"path": "message_body",
"path_part_infos": [
{
"key": "message_body",
"is_nested": false,
"anonymous_array_level": 0
}
],
"has_nested": true,
"parrent_column_unique_id": 4294967295
}
},
{
"unique_id": -1,
"name": "message_body.name",
"type": "STRING",
"is_key": false,
"aggregation": "NONE",
"is_nullable": true,
"length": 2147483647,
"index_length": 0,
"visible": true,
"result_is_nullable": false,
"column_path_info": {
"path": "message_body.name",
"path_part_infos": [
{
"key": "message_body",
"is_nested": false,
"anonymous_array_level": 0
},
{
"key": "name",
"is_nested": false,
"anonymous_array_level": 0
}
],
"has_nested": true,
"parrent_column_unique_id": 1
}
}
],
"num_short_key_columns": 2,
"num_rows_per_row_block": 1024,
"compress_kind": "COMPRESS_LZ4",
"next_column_unique_id": 2,
"is_in_memory": false,
"delete_sign_idx": -1,
"sequence_col_idx": -1,
"sort_type": "LEXICAL",
"sort_col_num": 2,
"compression_type": "LZ4F",
"schema_version": 0,
"disable_auto_compaction": false,
"index": [
{
"index_id": 10322,
"index_name": "idx_var",
"index_type": "INVERTED",
"col_unique_id": [
1
],
"index_suffix_name": "",
"properties": {
"lower_case": "true"
}
},
{
"index_id": 10322,
"index_name": "idx_var",
"index_type": "INVERTED",
"col_unique_id": [
1
],
"index_suffix_name": "message%5Fbody%2Ename",
"properties": {
"lower_case": "true"
}
}
],
"version_col_idx": -1,
"store_row_column": false,
"enable_single_replica_compaction": false,
"skip_write_index_on_load": false,
"inverted_index_storage_format": "V1"
},
"segments_overlap_pb": "NONOVERLAPPING"
},
{
"rowset_id": 0,
"partition_id": 10323,
"tablet_id": 10356,
"txn_id": 4,
"tablet_schema_hash": 420352195,
"rowset_type": "BETA_ROWSET",
"rowset_state": "VISIBLE",
"start_version": 4,
"end_version": 4,
"num_rows": 0,
"total_disk_size": 0,
"data_disk_size": 0,
"index_disk_size": 0,
"empty": true,
"load_id": {
"hi": 4384587840556650281,
"lo": -5519733373993529893
},
"creation_time": 1718380163,
"tablet_uid": {
"hi": 2470781828009500345,
"lo": 6867912887031541946
},
"num_segments": 0,
"rowset_id_v2": "020000000000003b39458223160034a874aead8a36248297",
"newest_write_timestamp": 1718380162,
"tablet_schema": {
"keys_type": "DUP_KEYS",
"column": [
{
"unique_id": 0,
"name": "message_time",
"type": "DATETIMEV2",
"is_key": true,
"aggregation": "NONE",
"is_nullable": true,
"precision": 18,
"frac": 0,
"length": 8,
"index_length": 8,
"visible": true,
"result_is_nullable": false
},
{
"unique_id": 1,
"name": "message_body",
"type": "VARIANT",
"is_key": true,
"aggregation": "NONE",
"is_nullable": true,
"precision": 0,
"frac": 0,
"length": 65537,
"index_length": 65537,
"visible": true,
"result_is_nullable": false
}
],
"num_short_key_columns": 2,
"num_rows_per_row_block": 1024,
"compress_kind": "COMPRESS_LZ4",
"next_column_unique_id": 2,
"is_in_memory": false,
"delete_sign_idx": -1,
"sequence_col_idx": -1,
"sort_type": "LEXICAL",
"sort_col_num": 2,
"compression_type": "LZ4F",
"schema_version": 0,
"disable_auto_compaction": false,
"index": [
{
"index_id": 10322,
"index_name": "idx_var",
"index_type": "INVERTED",
"col_unique_id": [
1
],
"index_suffix_name": "",
"properties": {
"lower_case": "true"
}
}
],
"version_col_idx": -1,
"store_row_column": false,
"enable_single_replica_compaction": false,
"skip_write_index_on_load": false,
"inverted_index_storage_format": "V1"
},
"segments_overlap_pb": "OVERLAPPING"
},
{
"rowset_id": 0,
"partition_id": 10323,
"tablet_id": 10356,
"txn_id": 5,
"tablet_schema_hash": 420352195,
"rowset_type": "BETA_ROWSET",
"rowset_state": "VISIBLE",
"start_version": 5,
"end_version": 5,
"num_rows": 0,
"total_disk_size": 0,
"data_disk_size": 0,
"index_disk_size": 0,
"empty": true,
"load_id": {
"hi": 1638530538051815834,
"lo": -9208128104113891803
},
"creation_time": 1718380191,
"tablet_uid": {
"hi": 2470781828009500345,
"lo": 6867912887031541946
},
"num_segments": 0,
"rowset_id_v2": "020000000000004639458223160034a874aead8a36248297",
"newest_write_timestamp": 1718380191,
"tablet_schema": {
"keys_type": "DUP_KEYS",
"column": [
{
"unique_id": 0,
"name": "message_time",
"type": "DATETIMEV2",
"is_key": true,
"aggregation": "NONE",
"is_nullable": true,
"precision": 18,
"frac": 0,
"length": 8,
"index_length": 8,
"visible": true,
"result_is_nullable": false
},
{
"unique_id": 1,
"name": "message_body",
"type": "VARIANT",
"is_key": true,
"aggregation": "NONE",
"is_nullable": true,
"precision": 0,
"frac": 0,
"length": 65537,
"index_length": 65537,
"visible": true,
"result_is_nullable": false
}
],
"num_short_key_columns": 2,
"num_rows_per_row_block": 1024,
"compress_kind": "COMPRESS_LZ4",
"next_column_unique_id": 2,
"is_in_memory": false,
"delete_sign_idx": -1,
"sequence_col_idx": -1,
"sort_type": "LEXICAL",
"sort_col_num": 2,
"compression_type": "LZ4F",
"schema_version": 0,
"disable_auto_compaction": false,
"index": [
{
"index_id": 10322,
"index_name": "idx_var",
"index_type": "INVERTED",
"col_unique_id": [
1
],
"index_suffix_name": "",
"properties": {
"lower_case": "true"
}
}
],
"version_col_idx": -1,
"store_row_column": false,
"enable_single_replica_compaction": false,
"skip_write_index_on_load": false,
"inverted_index_storage_format": "V1"
},
"segments_overlap_pb": "OVERLAPPING"
}
],
"in_restore_mode": false,
"tablet_uid": {
"hi": 2470781828009500345,
"lo": 6867912887031541946
},
"preferred_rowset_type": "BETA_ROWSET",
"tablet_type": "TABLET_TYPE_DISK",
"replica_id": 10358,
"enable_unique_key_merge_on_write": false,
"binlog_config": {
"enable": false,
"ttl_seconds": 86400,
"max_bytes": 9223372036854775807,
"max_history_nums": 9223372036854775807
},
"compaction_policy": "size_based",
"time_series_compaction_goal_size_mbytes": 1024,
"time_series_compaction_file_count_threshold": 2000,
"time_series_compaction_time_threshold_seconds": 3600,
"time_series_compaction_empty_rowsets_threshold": 5,
"time_series_compaction_level_threshold": 1
}

这是对应的表结构：

MySQL [testdb]> desc doris_test_10 all \G;
*************************** 1. row ***************************
IndexName: doris_test_10
IndexKeysType: DUP_KEYS
Field: message_time
Type: DATETIME
InternalType: DATETIMEV2(0)
Null: Yes
Key: true
Default: NULL
Extra:
Visible: true
DefineExpr:
WhereClause:
*************************** 2. row ***************************
IndexName:
IndexKeysType:
Field: message_body
Type: VARIANT
InternalType: VARIANT
Null: Yes
Key: true
Default: NULL
Extra:
Visible: true
DefineExpr:
WhereClause:
2 rows in set (0.00 sec)

表的创建命令：

show create table doris_test_10;

CREATE TABLE doris_test_10 (
message_time DATETIME NULL,
message_body VARIANT NULL,
INDEX idx_var (message_body) USING INVERTED COMMENT ''
) ENGINE=OLAP
DUPLICATE KEY(message_time, message_body)
COMMENT 'OLAP'
DISTRIBUTED BY HASH(message_time) BUCKETS 16
PROPERTIES (
"replication_allocation" = "tag.location.default: 2",
"min_load_replica_num" = "-1",
"is_being_synced" = "false",
"storage_medium" = "hdd",
"storage_format" = "V2",
"inverted_index_storage_format" = "V1",
"light_schema_change" = "true",
"disable_auto_compaction" = "false",
"enable_single_replica_compaction" = "false",
"group_commit_interval_ms" = "10000",
"group_commit_data_bytes" = "134217728"
);

Doris 2.1.3 BE启动后每24小时会自己停止

2 Answers