doris升级2.0.3后，catalog的元数据增量同步失败

Viewed 46
2.0

doris之前版本为1.2.7版本，最近升级到2.0.3版本后，发现之前配置的catalog元数据增量同步一直报错，导致新增的表无法及时通过catalog读取出来，只能refresh catalog才能读到，以下为fe.log日志报错内容：
2024-08-21 15:03:41,276 INFO (org.apache.doris.datasource.hive.event.MetastoreEventsProcessor|27) [MetastoreEventsProcessor.realRun():146] Events size are 1000 on catalog [hive_catalog]
2024-08-21 15:03:41,277 WARN (org.apache.doris.datasource.hive.event.MetastoreEventsProcessor|27) [MetastoreEventsProcessor.realRun():153] Failed to process hive metastore [hive_catalog] events .
java.lang.NullPointerException: null
at org.apache.doris.datasource.hive.event.MetastoreEvent.(MetastoreEvent.java:78) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.datasource.hive.event.IgnoredEvent.(IgnoredEvent.java:31) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.datasource.hive.event.IgnoredEvent.getEvents(IgnoredEvent.java:36) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.datasource.hive.event.MetastoreEventFactory.transferNotificationEventToMetastoreEvents(MetastoreEventFactory.java:70) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.datasource.hive.event.MetastoreEventFactory.getMetastoreEvents(MetastoreEventFactory.java:78) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.datasource.hive.event.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:117) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.datasource.hive.event.MetastoreEventsProcessor.realRun(MetastoreEventsProcessor.java:148) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.datasource.hive.event.MetastoreEventsProcessor.runAfterCatalogReady(MetastoreEventsProcessor.java:130) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.common.util.MasterDaemon.runOneCycle(MasterDaemon.java:58) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.common.util.Daemon.run(Daemon.java:116) ~[doris-fe.jar:1.2-SNAPSHOT]
2024-08-21 15:03:41,413 INFO (tablet scheduler|32) [BeLoadRebalancer.selectAlternativeTabletsForCluster():118] get number of low load paths: 2, with medium: HDD
2024-08-21 15:03:42,415 INFO (tablet scheduler|32) [BeLoadRebalancer.selectAlternativeTabletsForCluster():118] get number of low load paths: 2, with medium: HDD
2024-08-21 15:03:43,302 INFO (binlog-gcer|47) [BinlogManager.gc():359] begin gc binlog
2024-08-21 15:03:43,302 INFO (binlog-gcer|47) [BinlogManager.gc():370] gc binlog, dbBinlogMap is null

echo3
1 Answers

新增的表只能refresh后才能看到呀，1.2.7 如果不refresh应该也没办法看到新增的表的吧。

建议在catalog 参数中配置一个自动刷新试试的，还有版本这块，建议直接升级到最新的2.0.14上，2.0.3 这个版本也比较旧了，有些问题估计在后边的版本上已经fix过了。

比如：

-- 设置catalog刷新间隔为20秒
CREATE CATALOG es PROPERTIES (
    "type"="es",
    "hosts"="http://127.0.0.1:9200",
    "metadata_refresh_interval_sec"="20"
);
阿渊@SelectDB4853