表字段名中有特殊字符"@"时,where条件通过该字段delete数据后表打不开

Viewed 51

Doris版本1.2.7

表结构:

CREATE TABLE `ods_outwebtracklog_old` (
  `visit_date` bigint(20) NULL,
  `@timestamp` bigint(20) NULL,
  `idsite` text NULL,
  `vday` int(11) NULL,
  `vdayh` int(11) NULL,
  `device_Id` text NULL,
  `sid` text NULL,
  `is_new_session` tinyint(4) NULL,
  `is_new_user` tinyint(4) NULL,
  `visitor_days_since_first` tinyint(4) NULL,
  `visitor_days_since_last` tinyint(4) NULL,
  `page_title` text NULL,
  `page_url` text NULL,
  `page_detail_url` text NULL,
  `page_url_ref` text NULL,
  `page_url_ref_detail` text NULL,
  `page_espace_time` int(11) NULL,
  `prepage_espace_time` int(11) NULL,
  `event_type` tinyint(4) NULL,
  `event` jsonb NULL,
  `content` jsonb NULL,
  `location` jsonb NULL,
  `browse` jsonb NULL,
  `user_id` text NULL,
  `strackid` text NULL,
  `gtrackid` text NULL,
  `compaign` jsonb NULL,
  `pagevar` jsonb NULL,
  `uservar` jsonb NULL
) ENGINE=OLAP
DUPLICATE KEY(`visit_date`, `@timestamp`)
COMMENT 'OLAP'
DISTRIBUTED BY HASH(`visit_date`) BUCKETS AUTO
PROPERTIES (
"replication_allocation" = "tag.location.default: 3",
"in_memory" = "false",
"storage_format" = "V2",
"enable_unique_key_merge_on_write" = "true",
"light_schema_change" = "true",
"disable_auto_compaction" = "false"
);

执行如下delete语句后:

delete from ods_outwebtracklog_old where `@timestamp` >= unix_timestamp('2024-8-25')*1000

delete语句返回执行成功,但是查询该表报错:

errCode = 2, detailMessage = (172.16.24.65)[INTERNAL_ERROR]failed to initialize storage reader. tablet=144884.1497314462.2a4c870a19ffe927-cedd008e594e119f, res=[E-1903]
    @     0x5635da27aac4  doris::DeleteHandler::init()
    @     0x5635da0f1783  doris::TabletReader::_init_delete_condition()
    @     0x5635da0fb765  doris::TabletReader::_init_params()
    @     0x5635da0fbc7e  doris::TabletReader::init()
    @     0x5635df600fad  doris::vectorized::BlockReader::init()
    @     0x5635df80c627  doris::v

日志中有如下报错:

I0827 14:30:36.182729 17072 compaction.cpp:320] start vbase compaction. tablet=144884.1497314462.2a4c870a19ffe927-cedd008e594e119f, output_version=[0-12990739], permits: 34
W0827 14:30:36.183106 17072 delete_handler.cpp:260] fail to parse condition. [condition=@timestamp>=1724515200000]
W0827 14:30:36.231946 17072 reader.cpp:240] fail to init delete param. res = [E-1903]
    @     0x5635da27aac4  doris::DeleteHandler::init()
    @     0x5635da0f1783  doris::TabletReader::_init_delete_condition()
    @     0x5635da0fb765  doris::TabletReader::_init_params()
    @     0x5635da0fbc7e  doris::TabletReader::init()
    @     0x5635df600fad  doris::vectorized::BlockReader::init()
    @     0x5635da0ca6d2  doris::Merger::vmerge_rowsets()
    @     0x5635da0b8419  doris::Compaction::do_compaction_impl()
    @     0x5635da0b8bfd  doris::Compaction::do_compaction()
    @     0x5635da326d82  doris::BaseCompaction::execute_compact_impl()
    @     0x5635da0b3b05  doris::Compaction::execute_compact()
    @     0x5635da3969f4  doris::Tablet::execute_compaction()
    @     0x5635da06df3b  _ZNSt17_Function_handlerIFvvEZN5doris13StorageEngine23_submit_compaction_taskESt10shared_ptrINS1_6TabletEENS1_14CompactionTypeEEUlvE_E9_M_invokeERKSt9_Any_data
    @     0x5635daaf7af5  doris::ThreadPool::dispatch_thread()
    @     0x5635daaecf4f  doris::Thread::supervise_thread()
    @     0x7fceb35bcea5  start_thread
    @     0x7fceb38cf8dd  __clone
    @              (nil)  (unknown)
W0827 14:30:36.232000 17072 reader.cpp:95] fail to init reader when init params. res:[E-1903]
    @     0x5635da27aac4  doris::DeleteHandler::init()
    @     0x5635da0f1783  doris::TabletReader::_init_delete_condition()
    @     0x5635da0fb765  doris::TabletReader::_init_params()
    @     0x5635da0fbc7e  doris::TabletReader::init()
    @     0x5635df600fad  doris::vectorized::BlockReader::init()
    @     0x5635da0ca6d2  doris::Merger::vmerge_rowsets()
    @     0x5635da0b8419  doris::Compaction::do_compaction_impl()
    @     0x5635da0b8bfd  doris::Compaction::do_compaction()
    @     0x5635da326d82  doris::BaseCompaction::execute_compact_impl()
    @     0x5635da0b3b05  doris::Compaction::execute_compact()
    @     0x5635da3969f4  doris::Tablet::execute_compaction()
    @     0x5635da06df3b  _ZNSt17_Function_handlerIFvvEZN5doris13StorageEngine23_submit_compaction_taskESt10shared_ptrINS1_6TabletEENS1_14CompactionTypeEEUlvE_E9_M_invokeERKSt9_Any_data
    @     0x5635daaf7af5  doris::ThreadPool::dispatch_thread()
    @     0x5635daaecf4f  doris::Thread::supervise_thread()
    @     0x7fceb35bcea5  start_thread
    @     0x7fceb38cf8dd  __clone
    @              (nil)  (unknown)
, tablet_id:144884, schema_hash:1497314462, reader type:2, version:[0-12990739]
W0827 14:30:36.232030 17072 compaction.cpp:351] fail to do vbase compaction. res=[E-1903]
    @     0x5635da27aac4  doris::DeleteHandler::init()
    @     0x5635da0f1783  doris::TabletReader::_init_delete_condition()
    @     0x5635da0fb765  doris::TabletReader::_init_params()
    @     0x5635da0fbc7e  doris::TabletReader::init()
    @     0x5635df600fad  doris::vectorized::BlockReader::init()
    @     0x5635da0ca6d2  doris::Merger::vmerge_rowsets()
    @     0x5635da0b8419  doris::Compaction::do_compaction_impl()
    @     0x5635da0b8bfd  doris::Compaction::do_compaction()
    @     0x5635da326d82  doris::BaseCompaction::execute_compact_impl()
    @     0x5635da0b3b05  doris::Compaction::execute_compact()
    @     0x5635da3969f4  doris::Tablet::execute_compaction()
    @     0x5635da06df3b  _ZNSt17_Function_handlerIFvvEZN5doris13StorageEngine23_submit_compaction_taskESt10shared_ptrINS1_6TabletEENS1_14CompactionTypeEEUlvE_E9_M_invokeERKSt9_Any_data
    @     0x5635daaf7af5  doris::ThreadPool::dispatch_thread()
    @     0x5635daaecf4f  doris::Thread::supervise_thread()
    @     0x7fceb35bcea5  start_thread
    @     0x7fceb38cf8dd  __clone
    @              (nil)  (unknown)
W0827 14:30:36.232041 17072 tablet.cpp:1656] failed to do base compaction. res=[E-1903]
    @     0x5635da27aac4  doris::DeleteHandler::init()
    @     0x5635da0f1783  doris::TabletReader::_init_delete_condition()
    @     0x5635da0fb765  doris::TabletReader::_init_params()
    @     0x5635da0fbc7e  doris::TabletReader::init()
    @     0x5635df600fad  doris::vectorized::BlockReader::init()
    @     0x5635da0ca6d2  doris::Merger::vmerge_rowsets()
    @     0x5635da0b8419  doris::Compaction::do_compaction_impl()
    @     0x5635da0b8bfd  doris::Compaction::do_compaction()
    @     0x5635da326d82  doris::BaseCompaction::execute_compact_impl()
    @     0x5635da0b3b05  doris::Compaction::execute_compact()
    @     0x5635da3969f4  doris::Tablet::execute_compaction()
    @     0x5635da06df3b  _ZNSt17_Function_handlerIFvvEZN5doris13StorageEngine23_submit_compaction_taskESt10shared_ptrINS1_6TabletEENS1_14CompactionTypeEEUlvE_E9_M_invokeERKSt9_Any_data
    @     0x5635daaf7af5  doris::ThreadPool::dispatch_thread()
    @     0x5635daaecf4f  doris::Thread::supervise_thread()
    @     0x7fceb35bcea5  start_thread
    @     0x7fceb38cf8dd  __clone
    @              (nil)  (unknown)
, tablet=144884.1497314462.2a4c870a19ffe927-cedd008e594e119f
1 Answers

1、[E-1903]:表示删除参数无效;
2、'2024-8-25' 和 bigint类型不匹配,建议先减少@类的字段和对应类型的数据进行where筛选
3、后续可以考虑升级至2.0.14稳定版本或2.1.5相对稳定版本。