2.1.8 版本 异步物化视图 ALTER MATERIALIZED VIEW 后报错 unknown table ...

Viewed 73

ALTER MATERIALIZED VIEW reportview.dwd_fd_es_charge_owner_fee_mid_af REPLACE WITH MATERIALIZED VIEW dwd_fd_es_charge_owner_fee_mid_af_1
PROPERTIES('swap' = 'false');
采用这个命令去修改物化视图后,异步物化视图不再更新,查询FE日志发现,报错 unKnown table dwd_fd_es_charge_owner_fee_mid_af_1 这张表。

等待了一个小时左右,物化视图可以自动更新了,但是日志还是在报错 unKnown table dwd_fd_es_charge_owner_fee_mid_af_1

重启了 FE 后,不再报错 unKnown table dwd_fd_es_charge_owner_fee_mid_af_1
但是 各个视图 都会报错 collect insert target table ... more than once

CREATE MATERIALIZED VIEW dwd_fd_es_charge_owner_fee_mid_af
BUILD IMMEDIATE REFRESH AUTO ON SCHEDULE EVERY 10 MINUTE
DUPLICATE KEY(group_code)
DISTRIBUTED BY RANDOM BUCKETS 2
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"min_load_replica_num" = "-1",
"is_being_synced" = "false",
"storage_medium" = "hdd",
"storage_format" = "V2",
"inverted_index_storage_format" = "V1",
"light_schema_change" = "true",
"disable_auto_compaction" = "false",
"enable_single_replica_compaction" = "false",
"group_commit_interval_ms" = "10000",
"group_commit_data_bytes" = "134217728"
)
AS (select .....)

CREATE MATERIALIZED VIEW dwd_fd_es_charge_owner_fee_mid_af_1
BUILD IMMEDIATE REFRESH AUTO ON SCHEDULE EVERY 20 MINUTE
DUPLICATE KEY(group_code)
DISTRIBUTED BY RANDOM BUCKETS 2
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"min_load_replica_num" = "-1",
"is_being_synced" = "false",
"storage_medium" = "hdd",
"storage_format" = "V2",
"inverted_index_storage_format" = "V1",
"light_schema_change" = "true",
"disable_auto_compaction" = "false",
"enable_single_replica_compaction" = "false",
"group_commit_interval_ms" = "10000",
"group_commit_data_bytes" = "134217728"
)
AS (select .....)

上面两个是 物化视图的创建语句,由原来的10分钟更新一次,修改为 20分钟更新一次

ALTER MATERIALIZED VIEW reportview.dwd_fd_es_charge_owner_fee_mid_af REPLACE WITH MATERIALIZED VIEW dwd_fd_es_charge_owner_fee_mid_af_1
PROPERTIES('swap' = 'false');
执行完这个命令之后, 等待了20分钟,发现视图并没有更新,后续再没有做其他的操作. 只是在根据日志排查问题,大概过了 1个多小时左右 发现物化视图开始启动了,后续就正常跑了.

已经查询不到 当时的tasks了, 只保存了最近 100次的更新

4 Answers

collect insert target table more than once 这个报错是个警告日志,不影响物化的刷新

采用这个命令去修改物化视图后,异步物化视图不再更新,查询FE日志发现,报错 unKnown table dwd_fd_es_charge_owner_fee_mid_af_1 这张表。

这个报错的确可能是个问题,我们看下,如果需要细节,可能需要联系你下

看起来是三个问题点:
1.unKnown table dwd_fd_es_charge_owner_fee_mid_af_1这个日志是当前实现有些问题,但是对业务没有影响,可以先忽略,新的版本会修复
2.collect insert target table 这个没有影响,将来可能会修改日志打印规则
3.等待了一个小时左右,物化视图可以自动更新了。这个问题可以具体描述下么?粘一下物化视图的创建语句,通过tasks tvf看一下物化视图的任务列表