场景：
1、通过hms 挂载外表，表A
2、新建给予表A的异步物化视图a1
3、直接查询异步物化视图a1，耗时超长，主要耗时是Nereids Rewrite Time，见下方信息：
打开如下配置：
SET enable_nereids_planner = true;
SET enable_materialized_view_rewrite = true;
SET materialized_view_rewrite_enable_contain_external_table = true;
查询profile：
Summary:
- Profile ID: 41ee198019124afb-bed883bfb8a2be65
- Task Type: QUERY
- Start Time: 2024-07-26 06:26:09
- End Time: 2024-07-26 06:26:09
- Total: 25ms
- Task State: EOF
- User: root
- Default Db: adn_report
- Sql Statement: select * from table_a1 where pdate=20240725 limit 10
Execution Summary:
- Parse SQL Time: 0ms
- Nereids Analysis Time: 0ms
- Nereids Rewrite Time: -29574ms
- Nereids Optimize Time: 2ms
- Nereids Translate Time: N/A
- Workload Group: normal
- Analysis Time: 1ms
- Plan Time: 1ms
- JoinReorder Time: 1ms
- CreateSingleNode Time: 0ms
- QueryDistributed Time: 0ms
- Init Scan Node Time: N/A
- Finalize Scan Node Time: N/A
- Get Splits Time: N/A
- Get Partitions Time: N/A
- Get Partition Files Time: N/A
- Create Scan Range Time: N/A
- Schedule Time: 7ms
- Fragment Assign Time: 1ms
- Fragment Serialize Time: 0ms
- Fragment RPC Phase1 Time: 5ms
- Fragment RPC Phase2 Time: 1ms
- Fragment Compressed Size: 3.09 KB
- Fragment RPC Count: 2
- Wait and Fetch Result Time: 16ms
- Fetch Result Time: 14ms
- Write Result Time: 0ms
- Doris Version: doris-2.1.5-rc02-d5a02e095d
- Is Nereids: No
- Is Pipeline: Yes
- Is Cached: No
- Total Instances Num: 2
- Instances Num Per BE: 172.31.24.221:8060:2
- Parallel Fragment Exec Instance Num: 8
- Trace ID:
- Transaction Commit Time: N/A
MergedProfile
Fragments:
Fragment 0:
Pipeline : 0(instance_num=1):
RESULT_SINK_OPERATOR (id=0):
- PlanInfo
- TABLE: adn_report.table_a1(table_a1), PREAGGREGATION: ON
- PREDICATES: (`pdate` = 20240725)
- partitions=1/148 (p_20240725)
- tablets=1/1, tabletList=233148
- cardinality=10, avgRowSize=1033953.5, numNodes=3
- pushAggOp=NONE
- limit: 10
- BlocksProduced: sum 1, avg 1, max 1, min 1
- CloseTime: avg 3.114us, max 3.114us, min 3.114us
- ExecTime: avg 59.896us, max 59.896us, min 59.896us
- InitTime: avg 6.116us, max 6.116us, min 6.116us
- InputRows: sum 0, avg 0, max 0, min 0
- MemoryUsage: sum , avg , max , min
- PeakMemoryUsage: sum 0.00 , avg 0.00 , max 0.00 , min 0.00
- OpenTime: avg 12.671us, max 12.671us, min 12.671us
- RowsProduced: sum 10, avg 10, max 10, min 10
- WaitForDependencyTime: avg 0ns, max 0ns, min 0ns
- WaitForDependency[RESULT_SINK_OPERATOR_DEPENDENCY]Time: avg 0ns, max 0ns, min 0ns
EXCHANGE_OPERATOR (id=1):
- PlanInfo
- offset: 0
- limit: 10
- BlocksProduced: sum 1, avg 1, max 1, min 1
- CloseTime: avg 2.680us, max 2.680us, min 2.680us
- ExecTime: avg 29.690us, max 29.690us, min 29.690us
- InitTime: avg 22.478us, max 22.478us, min 22.478us
- MemoryUsage: sum , avg , max , min
- PeakMemoryUsage: sum 3.31 KB, avg 3.31 KB, max 3.31 KB, min 3.31 KB
- OpenTime: avg 1.503us, max 1.503us, min 1.503us
- ProjectionTime: avg 0ns, max 0ns, min 0ns
- RowsProduced: sum 10, avg 10, max 10, min 10
- WaitForDependencyTime: avg 0ns, max 0ns, min 0ns
- WaitForData0: avg 737.478us, max 737.478us, min 737.478us
Fragment 1:
Pipeline : 0(instance_num=1):
DATA_STREAM_SINK_OPERATOR (id=1,dst_id=1):
- BlocksProduced: sum 1, avg 1, max 1, min 1
- CloseTime: avg 6.677us, max 6.677us, min 6.677us
- ExecTime: avg 67.492us, max 67.492us, min 67.492us
- InitTime: avg 14.282us, max 14.282us, min 14.282us
- InputRows: sum 10, avg 10, max 10, min 10
- MemoryUsage: sum , avg , max , min
- PeakMemoryUsage: sum 0.00 , avg 0.00 , max 0.00 , min 0.00
- OpenTime: avg 22.407us, max 22.407us, min 22.407us
- RowsProduced: sum 10, avg 10, max 10, min 10
- WaitForDependencyTime: avg 0ns, max 0ns, min 0ns
- WaitForRpcBufferQueue: avg 0ns, max 0ns, min 0ns
OLAP_SCAN_OPERATOR (id=0. table name = table_a1(table_a1)):
- BlocksProduced: sum 1, avg 1, max 1, min 1
- CloseTime: avg 185.138us, max 185.138us, min 185.138us
- ExecTime: avg 950.253us, max 950.253us, min 950.253us
- InitTime: avg 77.317us, max 77.317us, min 77.317us
- MemoryUsage: sum , avg , max , min
- PeakMemoryUsage: sum 0.00 , avg 0.00 , max 0.00 , min 0.00
- OpenTime: avg 150.196us, max 150.196us, min 150.196us
- ProjectionTime: avg 0ns, max 0ns, min 0ns
- RowsProduced: sum 10, avg 10, max 10, min 10
- RuntimeFilterInfo: sum , avg , max , min
- WaitForDependency[OLAP_SCAN_OPERATOR_DEPENDENCY]Time: avg 433.3us, max 433.3us, min 433.3us
VScanner:
- MemoryUsage: sum , avg , max , min
- FreeBlocks: sum 1.72 MB, avg 1.72 MB, max 1.72 MB, min 1.72 MB
Execution Profile 41ee198019124afb-bed883bfb8a2be65:(Active: 25ms, % non-child: 0.00%)
Fragments:
Fragment 0:
Pipeline :0 (host=TNetworkAddress(hostname:172.31.24.221, port:9050)):
PipelineXTask (index=0):(Active: 155.980us, % non-child: 0.00%)
- CoreChangeTimes: 0
- ExecuteTime: 75.812us
- CloseTime: 6.909us
- GetBlockTime: 4.179us
- OpenTime: 18.337us
- PrepareTime: 51.280us
- SinkTime: 37.721us
- GetBlockCounter: 1
- NumBlockedBySinkTimes: 0
- NumBlockedBySrcTimes: 1
- NumBlockedTimes: 3
- NumScheduleTimes: 3
- NumYieldTimes: 0
- PendingFinishTimes: 0
- TaskCpuTime: 140.662us
- WaitBfTime: 0ns
- WaitBfTimes: 0
- WaitDenpendencyTimes: 1
- WaitWorkerTime: 34.899us
RESULT_SINK_OPERATOR (id=0):
- AppendBatchTime: 36.433us
- CopyBufferTime: 0ns
- ResultSendTime: 1.100us
- TupleConvertTime: 20.603us
- BlocksProduced: 1
- BytesSent: 958.00 B
- CloseTime: 3.114us
- ExecTime: 59.896us
- InitTime: 6.116us
- InputRows: 0
- MemoryUsage:
- PeakMemoryUsage: 0.00
- NumSentRows: 10
- OpenTime: 12.671us
- PendingFinishDependency: 0ns
- RowsProduced: 10
- WaitForDependencyTime: 0ns
- WaitForDependency[RESULT_SINK_OPERATOR_DEPENDENCY]Time: 0ns
EXCHANGE_OPERATOR (id=1):
- BlocksProduced: 1
- BytesReceived: 0.00
- CloseTime: 2.680us
- DataArrivalWaitTime: 0ns
- DecompressBytes: 0.00
- DecompressTime: 0ns
- DeserializeRowBatchTimer: 0ns
- ExecTime: 29.690us
- FirstBatchArrivalWaitTime: 0ns
- InitTime: 22.478us
- LocalBytesReceived: 3.44 KB
- MemoryUsage:
- Blocks: 3.44 KB
- PeakMemoryUsage: 3.31 KB
- OpenTime: 1.503us
- ProjectionTime: 0ns
- RowsProduced: 10
- SendersBlockedTotalTimer(*): 0ns
- WaitForDependencyTime: 0ns
- WaitForData0: 737.478us
Fragment 1:
Pipeline :0 (host=TNetworkAddress(hostname:172.31.24.221, port:9050)):
PipelineXTask (index=0):(Active: 821.96us, % non-child: 0.00%)
- CoreChangeTimes: 0
- ExecuteTime: 320.791us
- CloseTime: 192.763us
- GetBlockTime: 105.391us
- OpenTime: 176.224us
- PrepareTime: 128.927us
- SinkTime: 24.336us
- GetBlockCounter: 1
- NumBlockedBySinkTimes: 0
- NumBlockedBySrcTimes: 1
- NumBlockedTimes: 3
- NumScheduleTimes: 3
- NumYieldTimes: 0
- PendingFinishTimes: 0
- TaskCpuTime: 622.9us
- WaitBfTime: 0ns
- WaitBfTimes: 0
- WaitDenpendencyTimes: 1
- WaitWorkerTime: 31.829us
DATA_STREAM_SINK_OPERATOR (id=1,dst_id=1):
- BlocksProduced: 1
- BrpcSendTime: 0ns
- BrpcSendTime.Wait: 0ns
- BytesSent: 0.00
- CloseTime: 6.677us
- CompressTime: 0ns
- ExecTime: 67.492us
- InitTime: 14.282us
- InputRows: 10
- LocalBytesSent: 1.43 KB
- LocalSendTime: 20.320us
- LocalSentRows: 10
- MemoryUsage:
- PeakMemoryUsage: 0.00
- MergeBlockTime: 0ns
- OpenTime: 22.407us
- OverallThroughput: 0.0 /sec
- PendingFinishDependency: 0ns
- RowsProduced: 10
- RpcAvgTime: 0ns
- RpcCount: 0
- RpcMaxTime: 0ns
- RpcMinTime: 0ns
- RpcSumTime: 0ns
- SerializeBatchTime: 0ns
- SplitBlockDistributeByChannelTime: 0ns
- SplitBlockHashComputeTime: 0ns
- UncompressedRowBatchSize: 0.00
- WaitForDependencyTime: 0ns
- WaitForRpcBufferQueue: 0ns
OLAP_SCAN_OPERATOR (id=0. table name = table_a1(table_a1)):
- RuntimeFilters: :
- PushDownPredicates: []
- KeyRanges: ScanKeys:ScanKey=[20240725,null(-2147483648) : 20240725,2147483647]
- TabletIds: [233148]
- UseSpecificThreadToken: False
- AcquireRuntimeFilterTime: 644ns
- BlocksProduced: 1
- CloseTime: 185.138us
- ExecTime: 950.253us
- InitTime: 77.317us
- KeyRangesNum: 0
- MaxScannerThreadNum: 5
- MemoryUsage:
- PeakMemoryUsage: 0.00
- NumScanners: 5
- OpenTime: 150.196us
- ProcessConjunctTime: 65.170us
- ProjectionTime: 0ns
- RowsProduced: 10
- RowsRead: 16.256K (16256)
- RuntimeFilterInfo:
- ScannerWorkerWaitTime: 199.724us
- TabletNum: 1
- TotalReadThroughput: 0
- WaitForDependency[OLAP_SCAN_OPERATOR_DEPENDENCY]Time: 433.3us
- WaitForRuntimeFilter: 0ns
VScanner:
- ReadColumns: [pdate, ptimestamp, date, timestamp, platform, exp_id, rev, cost, click, install, impression
, bid, bid_price, pv, request]
- PerScannerRunningTime: [4.272us, 6.607us, 4.693us, 0.000ns, 4.945us, ]
- PerScannerRowsRead: [4.06K, 4.06K, 4.06K, 0, 4.06K, ]
- PerScannerWaitTime: [48.814us, 44.025us, 40.248us, 37.179us, 29.458us, ]
- PerScannerProjectionTime: [0.000ns, 0.000ns, 0.000ns, 0.000ns, 0.000ns, ]
- BlockConvertTime: 0ns
- BlockFetchTime: 0ns
- DeleteBitmapGetAggTime: 0ns
- MemoryUsage:
- FreeBlocks: 1.72 MB
- NewlyCreateFreeBlocksNum: 0
- NumScaleUpScanners: 0
- ReaderInitTime: 2.45ms
- RowsDelFiltered: 0
- ScannerBatchWaitTime: 37ns
- ScannerConvertBlockTime: 0ns
- ScannerCpuTime: 2.451ms
- ScannerCtxSchedTime: 190.262us
- ScannerFilterTime: 7.395us
- ScannerGetBlockTime: 10.821us
- ScannerInitTime: 30.484us
- ScannerPrefilterTime: 0ns
- ScannerSchedCount: 5
SegmentIterator:
- BitmapIndexFilterTimer: 714ns
- BlockConditionsFilteredBloomFilterTime: 0ns
- BlockConditionsFilteredDictTime: 0ns
- BlockConditionsFilteredTime: 4.132us
- BlockConditionsFilteredZonemapRuntimePredicateTime: 0ns
- BlockConditionsFilteredZonemapTime: 0ns
- BlockInitSeekCount: 45
- BlockInitSeekTime: 148.623us
- BlockInitTime: 227.974us
- BlockLoadTime: 1.606ms
- BlocksLoad: 5
- CachedPagesNum: 88
- CollectIteratorMergeTime: 0ns
- CollectIteratorNormalTime: 0ns
- CompressedBytesRead: 0.00
- DecompressorTimer: 0ns
- ExprFilterEvalTime: 0ns
- FirstReadSeekCount: 75
- FirstReadSeekTime: 173.411us
- FirstReadTime: 1.106ms
- IOTimer: 0ns
- InvertedIndexFilterTime: 1.24us
- InvertedIndexQueryBitmapCopyTime: 0ns
- InvertedIndexQueryBitmapOpTime: 0ns
- InvertedIndexQueryCacheHit: 0
- InvertedIndexQueryCacheMiss: 0
- InvertedIndexQueryTime: 0ns
- InvertedIndexSearcherOpenTime: 0ns
- InvertedIndexSearcherSearchTime: 0ns
- LazyReadSeekCount: 0
- LazyReadSeekTime: 0ns
- LazyReadTime: 0ns
- NumSegmentFiltered: 0
- NumSegmentTotal: 5
- OutputColumnTime: 2.383us
- OutputIndexResultColumnTimer: 774ns
- RawRowsRead: 20.32K (20320)
- RowsBitmapIndexFiltered: 0
- RowsBloomFilterFiltered: 0
- RowsConditionsFiltered: 0
- RowsDictFiltered: 0
- RowsInvertedIndexFiltered: 0
- RowsKeyRangeFiltered: 0
- RowsShortCircuitPredFiltered: 0
- RowsShortCircuitPredInput: 0
- RowsStatsFiltered: 0
- RowsVectorPredFiltered: 0
- RowsVectorPredInput: 0
- RowsZonemapRuntimePredicateFiltered: 0
- SecondReadTime: 0ns
- ShortPredEvalTime: 0ns
- TotalPagesNum: 88
- UncompressedBytesRead: 0.00
- VectorPredEvalTime: 0ns
LoadChannels:
