publish version 过多 引起计算错误

Viewed 52
2.0

版本2.0.13
publish version 过多引起数据指标计算错误
目前因为publish version 问题，已经反复出现多次了，当模型之间有依赖关系时，出现此问题，就会导致下游计算全部错误！请教各位大佬，有没有办法可以避免这种问题？

yky_young61
2 Answers

日志确认是FE publish慢了，gc看起来挺频繁的。调整jvm内存和insert_visible_timeout_ms之后，积压数量和频率没有之前那么多了，暂时没出现之前那种问题。

徐振超@SelectDB5016

日志补充

I0910 06:16:44.194433 15766 tablet.cpp:3326] skip to construct delete bitmap tablet: 9902119 rowset: 020000001644251fe846031b097cc1c65bb0120f5a9fc9a5
I0910 06:16:44.194443 15766 tablet.cpp:3715] [Publish] construct delete bitmap tablet: 9902119, rowset_ids to add: 0, rowset_ids to del: 0, cur version: 1227, transaction_id: 133788686,cost: 13(us) , total rows: 110
I0910 06:16:44.194563 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788708|queue_size=3
I0910 06:16:44.194588 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788642|queue_size=3
I0910 06:16:44.194602 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788644|queue_size=4
I0910 06:16:44.194609 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788643|queue_size=5
I0910 06:16:44.194787 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788647|queue_size=4
I0910 06:16:44.194851 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788646|queue_size=3
I0910 06:16:44.194861 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788645|queue_size=4
I0910 06:16:44.194868 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788648|queue_size=5
I0910 06:16:44.194885 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788652|queue_size=6
I0910 06:16:44.194891 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788653|queue_size=7
I0910 06:16:44.194898 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788655|queue_size=8
I0910 06:16:44.194909 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788654|queue_size=9
I0910 06:16:44.194916 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788657|queue_size=10
I0910 06:16:44.194922 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788658|queue_size=11
I0910 06:16:44.194929 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788656|queue_size=12
I0910 06:16:44.194944 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788659|queue_size=13
I0910 06:16:44.194952 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788661|queue_size=14
I0910 06:16:44.194958 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788660|queue_size=15
I0910 06:16:44.194963 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788663|queue_size=16
I0910 06:16:44.194969 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788664|queue_size=17
I0910 06:16:44.194980 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788665|queue_size=18
I0910 06:16:44.195020 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788662|queue_size=19
I0910 06:16:44.195027 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788667|queue_size=20
I0910 06:16:44.195034 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788666|queue_size=21
I0910 06:16:44.195048 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788668|queue_size=22
I0910 06:16:44.195055 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788669|queue_size=23
I0910 06:16:44.195065 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788671|queue_size=24
I0910 06:16:44.195073 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788672|queue_size=25
I0910 06:16:44.195079 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788674|queue_size=26
I0910 06:16:44.195088 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788673|queue_size=27
I0910 06:16:44.195094 15767 tablet.cpp:3326] skip to construct delete bitmap tablet: 14284781 rowset: 020000001644250ee846031b097cc1c65bb0120f5a9fc9a5
I0910 06:16:44.195116 15767 tablet.cpp:3715] [Publish] construct delete bitmap tablet: 14284781, rowset_ids to add: 0, rowset_ids to del: 0, cur version: 1047, transaction_id: 133788689,cost: 31(us) , total rows: 5
I0910 06:16:44.195094 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788675|queue_size=28
I0910 06:16:44.195129 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788676|queue_size=29
I0910 06:16:44.195137 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788677|queue_size=30
I0910 06:16:44.195142 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788678|queue_size=31
I0910 06:16:44.195145 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788680|queue_size=32
I0910 06:16:44.195150 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788681|queue_size=33
I0910 06:16:44.195155 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788682|queue_size=34
I0910 06:16:44.195160 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788683|queue_size=35
I0910 06:16:44.195164 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788684|queue_size=36
I0910 06:16:44.195168 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788685|queue_size=37
I0910 06:16:44.195173 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788697|queue_size=38
I0910 06:16:44.195195 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788698|queue_size=39
I0910 06:16:44.195202 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788699|queue_size=40
I0910 06:16:44.195214 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788700|queue_size=41
I0910 06:16:44.195220 232871 task_worker_pool.cpp:264] successfully submit task|type=PUBLISH_VERSION|signature=133788702|queue_size=42
yky_young61