hive的catalog查询表数据为空

Viewed 88

集群架构四台节点,版本:1.2.6,分别是1fe,3be,1observer,1observer和一台be混布,然后最近一直出现通过master的fe读取catalog的表有时候读不出来数据,读取表数据为空,但是通过Observer是可以读取出来表数据的,我看了fe和observer的fe.conf配置是一样的,都配置了增量的元数据同步,参数如下:
enable_hms_events_incremental_sync = true
hms_events_polling_interval_ms = 60000
hms_events_batch_size_per_rpc = 10000
HADOOP_USER_NAME = hive
max_broker_concurrency = 3
master节点的fe如果查询不到表数据,此时如果刷新一下catalog,master的fe节点就会恢复查询正常,日志未看到明显的报错信息。望社区同学能够给予帮助!!

2 Answers

在hive catalog添加如下属性

'file.meta.cache.ttl-second' = '0',
'metadata_refresh_interval_sec' = '60'

HIVE版本为:3.1.0;
以下是创建语句:
CREATE CATALOG hive_catalog PROPERTIES (
"specified_database_list" = "edw",
"dfs.namenode.rpc-address.eccomCluster.nn2" = "xxxxxx:8020",
"dfs.namenode.rpc-address.eccomCluster.nn1" = "xxxxxx:8020",
"hive.version" = "3.1.0",
"dfs.client.failover.proxy.provider.eccomCluster" = "org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider",
"dfs.ha.namenodes.eccomCluster" = "nn1,nn2",
"hive.metastore.uris" = "thrift://xxxxxxxx:9083",
"type" = "hms",
"dfs.nameservices" = "xxxxx",
"hadoop.username" = "hive"
);