Doris在做压测时，出现查询效率低的问题，
基本信息如下：
Doris三台机器，1台FE，三台BE，64核，128G，HDD
单表600W，大约1G，分两个桶，没有分区，SQL语句单条查询大概在1.5-2秒左右，压测100并发时间在13-20秒之间
QueryProfile分析：
根据QueryProfile信息，
1. 在Instance 26d0866efab94717-a8ef244b11cb8f1d(.224)的PipelineTask (index=1)中Task1BeginExecuteTime，Task2EosTime 达到18S，WaitWorkerTime达到10S
2. 在Instance 26d0866efab94717-a8ef244b11cb8f1e(.223)的PipelineTask (index=1)中Task1BeginExecuteTime，Task2EosTime 达到6S，WaitWorkerTime达到1S
3. 两台节点配置一样，均只部署了BE，数据量差不多
4. 经过分析，发现最终就是在上述的任务处理耗时，导致真个查询效率低下
问题：
1. 请帮忙分析性能低下的可能原因，是Doris配置问题，还是服务器问题
2. 对应的QueyrProfile，有没有官方标准的说明文档，当前queryprofile的很多属性在官方都没有给出明确的说明
Query:
Summary:
- Profile ID: 26d0866efab94717-a8ef244b11cb8f1c
- Task Type: QUERY
- Start Time: 2024-08-16 11:27:16
- End Time: 2024-08-16 11:27:34
- Total: 18s309ms
- Task State: EOF
- User: default_cluster:readonly_user
- Default Db: default_cluster:hive
- Sql Statement: select A,B,C,D
,sum(F1) as F1
,sum(F2) as F2
,sum(F3) as F3
,sum(F4) as F4
,sum(F5) as F5
,sum(F6) as F6
,sum(F7) as F7
,sum(F8) as F8
,sum(F9) as F9
,sum(F10) as F10
,sum(F11) as F11
,sum(F12) as F12
,sum(F13) as F13
,sum(F14) as F14
,sum(F15) as F15
,sum(F16) as F16
,sum(F17) as F17
from MAIN_TABLE
WHERE F18 >= '2023-07-06'
AND F18 <= '2024-07-29'
group by A,B,C,D
order by A LIMIT 2628,6
Execution Summary:
- Workload Group:
- Analysis Time: 2ms
- Plan Time: 5ms
- JoinReorder Time: N/A
- CreateSingleNode Time: N/A
- QueryDistributed Time: N/A
- Init Scan Node Time: N/A
- Finalize Scan Node Time: N/A
- Get Splits Time: N/A
- Get PARTITIONS Time: N/A
- Get PARTITION FILES Time: N/A
- Create Scan Range Time: N/A
- Schedule Time: 67ms
- Fetch Result Time: 18s235ms
- Write Result Time: 0ms
- Wait and Fetch Result Time: 18s235ms
- Doris Version: doris-2.0.3-rc06-37d31a5
- Is Nereids: Yes
- Is Pipeline: Yes
- Is Cached: No
- Total Instances Num: 5
- Instances Num Per BE: 192.168.30.223:8060:2,192.168.30.224:8060:3
- Parallel Fragment Exec Instance Num: 16
- Trace ID:
Execution Profile 26d0866efab94717-a8ef244b11cb8f1c:(Active: 18s309ms, % non-child: 0.00%)
Fragments:
Fragment 0:
Instance 26d0866efab94717-a8ef244b11cb8f21 (host=TNetworkAddress(hostname:192.168.30.224, port:9060)):
VDataBufferSender (dst_fragment_instance_id=26d0866efab94717--5710dbb4ee3470df):
- AppendBatchTime: 243.58us
- CopyBufferTime: 0ns
- ResultSendTime: 136.935us
- TupleConvertTime: 68.525us
- BytesSent: 1.47 KB
- NumSentRows: 6
VSORT_NODE (id=1186):(Active: 82.754us, % non-child: 0.00%)
- TOP-N: true
- Spilled: false
- ChildGetResultTime: 0ns
- GetResultTime: 108.174us
- MemoryUsage:
- PeakMemoryUsage: 0.00
- SortBlocks: 0.00
- MergeBlockTime: 14.973ms
- PartialSortTime: 6.804ms
- PartialSortTotalTime: 0ns
- ProjectionTime: 0ns
- RowsReturned: 6
- RowsReturnedRate: 72.504K /sec
BlockSpill:
- BlockBytes: 0.00
- BlockCount: 0
VEXCHANGE_NODE (id=1183):(Active: 839.951us, % non-child: 0.00%)
- BlocksProduced: 7
- BytesReceived: 2.59 MB
- DataArrivalWaitTime: 0ns
- DecompressBytes: 9.27 MB
- DecompressTime: 6.333ms
- DeserializeRowBatchTimer: 10.590ms
- FirstBatchArrivalWaitTime: 0ns
- LocalBytesReceived: 14.12 MB
- MemoryUsage:
- Blocks: 10.17 MB
- PeakMemoryUsage: 3.35 KB
- ProjectionTime: 0ns
- RowsProduced: 24.986K (24986)
- RowsReturned: 50.104K (50104)
- RowsReturnedRate: 59.651098M /sec
- SendersBlockedTotalTimer(*): 0ns
PipelineContext:(Active: 18s264ms, % non-child: 0.00%)
- PrepareTime: 511.704us
- StartTime: 5.503us
Pipeline (pipeline id=0):
PipelineTask (index=0):(Active: 517.630us, % non-child: 0.00%)
- Sink: ResultSinkOperator(dst_id=10000)
- OperatorIds(source2root): [SortSourceOperator(node_id=1186)]
- CoreChangeTimes: 1
- ExecuteTime: 380.737us
- CloseTime: 102.253us
- FinalizeTime: 218ns
- GetBlockTime: 110.369us
- OpenTime: 8.700us
- PrepareTime: 7.11us
- SinkTime: 249.763us
- GetBlockCounter: 1
- NumBlockedBySinkTimes: 0
- NumBlockedBySrcTimes: 1
- NumBlockedTimes: 2
- NumScheduleTimes: 2
- NumYieldTimes: 0
- Task1BeginExecuteTime: 18s263ms
- Task2EosTime: 18s264ms
- Task3SrcPendingFinishOverTime: 18s264ms
- Task4DstPendingFinishOverTime: 18s264ms
- Task5TotalTime: 18s264ms
- Task6ClosePipelineTime: 18s264ms
- TaskCpuTime: 406.217us
- WaitBfTime: 0ns
- WaitSinkTime: 0ns
- WaitSourceTime: 18s185ms
- WaitWorkerTime: 78.205ms
Pipeline (pipeline id=1):
PipelineTask (index=1):(Active: 23.66ms, % non-child: 0.00%)
- Sink: SortSinkOperator(dst_id=1186)
- OperatorIds(source2root): [ExchangeSourceOperator(node_id=1183)]
- CoreChangeTimes: 2
- ExecuteTime: 22.988ms
- CloseTime: 15.644us
- FinalizeTime: 434ns
- GetBlockTime: 876.601us
- OpenTime: 10.812us
- PrepareTime: 5.825us
- SinkTime: 21.948ms
- GetBlockCounter: 15
- NumBlockedBySinkTimes: 0
- NumBlockedBySrcTimes: 5
- NumBlockedTimes: 6
- NumScheduleTimes: 6
- NumYieldTimes: 0
- Task1BeginExecuteTime: 18s209ms
- Task2EosTime: 18s263ms
- Task3SrcPendingFinishOverTime: 18s263ms
- Task4DstPendingFinishOverTime: 18s263ms
- Task5TotalTime: 18s263ms
- Task6ClosePipelineTime: 18s263ms
- TaskCpuTime: 22.960ms
- WaitBfTime: 0ns
- WaitSinkTime: 0ns
- WaitSourceTime: 18s165ms
- WaitWorkerTime: 75.326ms
Fragment 1:
Instance 26d0866efab94717-a8ef244b11cb8f1f (host=TNetworkAddress(hostname:192.168.30.224, port:9060)):
VDataStreamSender (dst_id=1183, dst_fragments=[{"26d0866efab94717-a8ef244b11cb8f21"}]):(Active: 4.545ms, % non-child: 0.00%)
- BlocksSent: 7
- BrpcSendTime: 0ns
- BrpcSendTime.Wait: 0ns
- BytesSent: 0.00
- CompressTime: 0ns
- IgnoreRows: 0
- LocalBytesSent: 9.32 MB
- LocalSendTime: 4.494ms
- LocalSentRows: 25.118K (25118)
- MemoryUsage:
- PeakMemoryUsage: 3.88 KB
- OverallThroughput: 0.0 /sec
- RpcAvgTime: 0ns
- RpcCount: 0
- RpcMaxTime: 0ns
- RpcMinTime: 2562047h47m
- RpcSumTime: 0ns
- SerializeBatchTime: 0ns
- SplitBlockDistributeByChannelTime: 0ns
- SplitBlockHashComputeTime: 0ns
- UncompressedRowBatchSize: 0.00
VAGGREGATION_NODE (id=1180):(Active: 148.834us, % non-child: 0.00%)
- AggInfos: (_is_merge: true, _needs_finalize: true, Streaming Preaggregation: false, agg size: 17, limit: -1)
- BuildConvertToPartitionedTime: 0ns
- DeserializeAndMergeTime: 61.919ms
- ExecTime: 164.573ms
- ExprTime: 0ns
- GetResultsTime: 38.239ms
- HashTableComputeTime: 63.297ms
- HashTableInputCount: 48.778K (48778)
- HashTableIterateTime: 262.481us
- HashTableSize: 25.118K (25118)
- InsertKeysToColumnTime: 3.999ms
- MaxRowSizeInBytes: 175
- MemoryUsage:
- HashTable: 767.98 KB
- PeakMemoryUsage: 33.78 MB
- SerializeKeyArena: 33.71 MB
- MergeTime: 125.868ms
- ProjectionTime: 0ns
- RowsReturned: 25.118K (25118)
- RowsReturnedRate: 168.765201M /sec
- SerializeDataTime: 0ns
- SerializeKeyTime: 11.83ms
- SerializeResultTime: 0ns
- StreamingAggTime: 0ns
VEXCHANGE_NODE (id=1177):(Active: 995.553us, % non-child: 0.00%)
- BlocksProduced: 6
- BytesReceived: 4.21 MB
- DataArrivalWaitTime: 0ns
- DecompressBytes: 10.63 MB
- DecompressTime: 9.42ms
- DeserializeRowBatchTimer: 13.752ms
- FirstBatchArrivalWaitTime: 0ns
- LocalBytesReceived: 17.71 MB
- MemoryUsage:
- Blocks: 17.63 MB
- PeakMemoryUsage: 3.35 KB
- ProjectionTime: 0ns
- RowsProduced: 24.383K (24383)
- RowsReturned: 48.778K (48778)
- RowsReturnedRate: 48.995884M /sec
- SendersBlockedTotalTimer(*): 0ns
PipelineContext:(Active: 18s247ms, % non-child: 0.00%)
- PrepareTime: 528.268us
- StartTime: 2.307us
Pipeline (pipeline id=0):
PipelineTask (index=0):(Active: 45.235ms, % non-child: 0.00%)
- Sink: ExchangeSinkOperator(dst_id=1183)
- OperatorIds(source2root): [AggSourceOperator(node_id=1180)]
- CoreChangeTimes: 1
- ExecuteTime: 43.323ms
- CloseTime: 1.858ms
- FinalizeTime: 360ns
- GetBlockTime: 38.658ms
- OpenTime: 18.313us
- PrepareTime: 9.774us
- SinkTime: 4.564ms
- GetBlockCounter: 7
- NumBlockedBySinkTimes: 0
- NumBlockedBySrcTimes: 1
- NumBlockedTimes: 2
- NumScheduleTimes: 2
- NumYieldTimes: 0
- Task1BeginExecuteTime: 18s202ms
- Task2EosTime: 18s245ms
- Task3SrcPendingFinishOverTime: 18s245ms
- Task4DstPendingFinishOverTime: 18s245ms
- Task5TotalTime: 18s247ms
- Task6ClosePipelineTime: 18s247ms
- TaskCpuTime: 37.128ms
- WaitBfTime: 0ns
- WaitSinkTime: 0ns
- WaitSourceTime: 18s26ms
- WaitWorkerTime: 175.451ms
Pipeline (pipeline id=1):
PipelineTask (index=1):(Active: 127.318ms, % non-child: 0.00%)
- Sink: AggSinkOperator(dst_id=1180)
- OperatorIds(source2root): [ExchangeSourceOperator(node_id=1177)]
- CoreChangeTimes: 3
- ExecuteTime: 127.182ms
- CloseTime: 32.597us
- FinalizeTime: 174ns
- GetBlockTime: 1.48ms
- OpenTime: 24.836us
- PrepareTime: 3.839us
- SinkTime: 125.940ms
- GetBlockCounter: 14
- NumBlockedBySinkTimes: 0
- NumBlockedBySrcTimes: 7
- NumBlockedTimes: 8
- NumScheduleTimes: 8
- NumYieldTimes: 0
- Task1BeginExecuteTime: 7s384ms
- Task2EosTime: 18s202ms
- Task3SrcPendingFinishOverTime: 18s202ms
- Task4DstPendingFinishOverTime: 18s202ms
- Task5TotalTime: 18s202ms
- Task6ClosePipelineTime: 18s202ms
- TaskCpuTime: 94.989ms
- WaitBfTime: 0ns
- WaitSinkTime: 0ns
- WaitSourceTime: 17s635ms
- WaitWorkerTime: 439.391ms
Instance 26d0866efab94717-a8ef244b11cb8f20 (host=TNetworkAddress(hostname:192.168.30.223, port:9060)):
VDataStreamSender (dst_id=1183, dst_fragments=[{"26d0866efab94717-a8ef244b11cb8f21"}]):(Active: 15.859ms, % non-child: 0.00%)
- BlocksSent: 8
- BrpcSendTime: 0ns
- BrpcSendTime.Wait: 0ns
- BytesSent: 2.59 MB
- CompressTime: 61.201ms
- IgnoreRows: 0
- LocalBytesSent: 0.00
- LocalSendTime: 0ns
- LocalSentRows: 0
- MemoryUsage:
- PeakMemoryUsage: 4.39 MB
- OverallThroughput: 163.0959644317627 MB/sec
- RpcAvgTime: 3.794ms
- RpcCount: 7
- RpcMaxTime: 26.558ms
- RpcMinTime: 26.558ms
- RpcSumTime: 26.558ms
- SerializeBatchTime: 15.536ms
- SplitBlockDistributeByChannelTime: 0ns
- SplitBlockHashComputeTime: 0ns
- UncompressedRowBatchSize: 9.27 MB
VAGGREGATION_NODE (id=1180):(Active: 665.498us, % non-child: 0.00%)
- AggInfos: (_is_merge: true, _needs_finalize: true, Streaming Preaggregation: false, agg size: 17, limit: -1)
- BuildConvertToPartitionedTime: 0ns
- DeserializeAndMergeTime: 33.486ms
- ExecTime: 91.808ms
- ExprTime: 0ns
- GetResultsTime: 10.922ms
- HashTableComputeTime: 46.736ms
- HashTableInputCount: 48.516K (48516)
- HashTableIterateTime: 116.109us
- HashTableSize: 24.986K (24986)
- InsertKeysToColumnTime: 2.12ms
- MaxRowSizeInBytes: 175
- MemoryUsage:
- HashTable: 767.98 KB
- PeakMemoryUsage: 33.78 MB
- SerializeKeyArena: 33.71 MB
- MergeTime: 80.643ms
- ProjectionTime: 0ns
- RowsReturned: 24.986K (24986)
- RowsReturnedRate: 37.544816M /sec
- SerializeDataTime: 0ns
- SerializeKeyTime: 7.79ms
- SerializeResultTime: 0ns
- StreamingAggTime: 0ns
VEXCHANGE_NODE (id=1177):(Active: 719.324us, % non-child: 0.00%)
- BlocksProduced: 6
- BytesReceived: 4.20 MB
- DataArrivalWaitTime: 0ns
- DecompressBytes: 10.57 MB
- DecompressTime: 4.435ms
- DeserializeRowBatchTimer: 6.627ms
- FirstBatchArrivalWaitTime: 0ns
- LocalBytesReceived: 17.63 MB
- MemoryUsage:
- Blocks: 5.88 MB
- PeakMemoryUsage: 3.35 KB
- ProjectionTime: 0ns
- RowsProduced: 24.252K (24252)
- RowsReturned: 48.516K (48516)
- RowsReturnedRate: 67.446658M /sec
- SendersBlockedTotalTimer(*): 0ns
PipelineContext:(Active: 18s274ms, % non-child: 0.00%)
- PrepareTime: 1.253ms
- StartTime: 12.56us
Pipeline (pipeline id=0):
PipelineTask (index=0):(Active: 27.844ms, % non-child: 0.00%)
- Sink: ExchangeSinkOperator(dst_id=1183)
- OperatorIds(source2root): [AggSourceOperator(node_id=1180)]
- CoreChangeTimes: 0
- ExecuteTime: 26.934ms
- CloseTime: 861.946us
- FinalizeTime: 99ns
- GetBlockTime: 11.89ms
- OpenTime: 20.744us
- PrepareTime: 12.807us
- SinkTime: 15.795ms
- GetBlockCounter: 7
- NumBlockedBySinkTimes: 0
- NumBlockedBySrcTimes: 1
- NumBlockedTimes: 2
- NumScheduleTimes: 3
- NumYieldTimes: 0
- Task1BeginExecuteTime: 18s224ms
- Task2EosTime: 18s251ms
- Task3SrcPendingFinishOverTime: 18s251ms
- Task4DstPendingFinishOverTime: 18s272ms
- Task5TotalTime: 18s273ms
- Task6ClosePipelineTime: 18s273ms
- TaskCpuTime: 26.975ms
- WaitBfTime: 0ns
- WaitSinkTime: 0ns
- WaitSourceTime: 18s148ms
- WaitWorkerTime: 76.182ms
Pipeline (pipeline id=1):
PipelineTask (index=1):(Active: 81.786ms, % non-child: 0.00%)
- Sink: AggSinkOperator(dst_id=1180)
- OperatorIds(source2root): [ExchangeSourceOperator(node_id=1177)]
- CoreChangeTimes: 1
- ExecuteTime: 81.645ms
- CloseTime: 5.313us
- FinalizeTime: 73ns
- GetBlockTime: 749.882us
- OpenTime: 53.309us
- PrepareTime: 3.28us
- SinkTime: 80.702ms
- GetBlockCounter: 13
- NumBlockedBySinkTimes: 0
- NumBlockedBySrcTimes: 10
- NumBlockedTimes: 12
- NumScheduleTimes: 12
- NumYieldTimes: 0
- Task1BeginExecuteTime: 172.986ms
- Task2EosTime: 18s224ms
- Task3SrcPendingFinishOverTime: 18s224ms
- Task4DstPendingFinishOverTime: 18s224ms
- Task5TotalTime: 18s224ms
- Task6ClosePipelineTime: 18s224ms
- TaskCpuTime: 70.716ms
- WaitBfTime: 0ns
- WaitSinkTime: 0ns
- WaitSourceTime: 17s731ms
- WaitWorkerTime: 395.885ms
Fragment 2:
Instance 26d0866efab94717-a8ef244b11cb8f1e (host=TNetworkAddress(hostname:192.168.30.223, port:9060)):
VDataStreamSender (dst_id=1177, dst_fragments=[{"26d0866efab94717-a8ef244b11cb8f1f", "26d0866efab94717-a8ef244b11cb8f20"}]):(Active: 78.996ms, % non-child: 0.00%)
- BlocksSent: 12
- BrpcSendTime: 0ns
- BrpcSendTime.Wait: 0ns
- BytesSent: 4.21 MB
- CompressTime: 25.972ms
- IgnoreRows: 0
- LocalBytesSent: 10.57 MB
- LocalSendTime: 290.151us
- LocalSentRows: 24.264K (24264)
- MemoryUsage:
- PeakMemoryUsage: 25.53 MB
- OverallThroughput: 53.2987699508667 MB/sec
- RpcAvgTime: 9.677ms
- RpcCount: 6
- RpcMaxTime: 58.62ms
- RpcMinTime: 58.62ms
- RpcSumTime: 58.62ms
- SerializeBatchTime: 32.130ms
- SplitBlockDistributeByChannelTime: 45.955ms
- SplitBlockHashComputeTime: 2.883ms
- UncompressedRowBatchSize: 10.63 MB
VAGGREGATION_NODE (id=1174):(Active: 502.281us, % non-child: 0.00%)
- AggInfos: (_is_merge: false, _needs_finalize: false, Streaming Preaggregation: true, agg size: 17, limit: -1)
- BuildConvertToPartitionedTime: 0ns
- DeserializeAndMergeTime: 0ns
- ExecTime: 4s897ms
- ExprTime: 35.910ms
- GetResultsTime: 87.349ms
- HashTableComputeTime: 2s154ms
- HashTableInputCount: 3.243522M (3243522)
- HashTableIterateTime: 506.115us
- HashTableSize: 48.647K (48647)
- InsertKeysToColumnTime: 7.175ms
- MaxRowSizeInBytes: 175
- MaxSizeInBlockQueue: 0.00
- MaxSizeOfBlockQueue: 0
- MemoryUsage:
- HashTable: 1.50 MB
- PeakMemoryUsage: 0.00
- SerializeKeyArena: 70.74 MB
- MergeTime: 0ns
- ProjectionTime: 0ns
- RowsReturned: 48.647K (48647)
- RowsReturnedRate: 96.85216M /sec
- SerializeDataTime: 79.391ms
- SerializeKeyTime: 793.985ms
- SerializeResultTime: 87.487ms
- StreamingAggTime: 0ns
VNewOlapScanNode(MAIN_TABLE) (id=1164):(Active: 11.751ms, % non-child: 0.00%)
- RuntimeFilters: :
- PushDownPredicates: []
- KeyRanges: ScanKeys:ScanKey=[2023-07-06 : 2023-07-14]ScanKey=[2023-07-15 : 2023-07-23]ScanKey=[2023-07-24 : 2023-08-01]ScanKey=[2023-08-02 : 2023-08-10]ScanKey=[2023-08-11 : 2023-08-19]ScanKey=[2023-08-20 : 2023-08-28]ScanKey=[2023-08-29 : 2023-09-06]ScanKey=[2023-09-07 : 2023-09-15]ScanKey=[2023-09-16 : 2023-09-24]ScanKey=[2023-09-25 : 2023-10-03]ScanKey=[2023-10-04 : 2023-10-12]ScanKey=[2023-10-13 : 2023-10-21]ScanKey=[2023-10-22 : 2023-10-30]ScanKey=[2023-10-31 : 2023-11-08]ScanKey=[2023-11-09 : 2023-11-17]ScanKey=[2023-11-18 : 2023-11-26]ScanKey=[2023-11-27 : 2023-12-05]ScanKey=[2023-12-06 : 2023-12-14]ScanKey=[2023-12-15 : 2023-12-23]ScanKey=[2023-12-24 : 2024-01-01]ScanKey=[2024-01-02 : 2024-01-10]ScanKey=[2024-01-11 : 2024-01-19]ScanKey=[2024-01-20 : 2024-01-28]ScanKey=[2024-01-29 : 2024-02-06]ScanKey=[2024-02-07 : 2024-02-15]ScanKey=[2024-02-16 : 2024-02-24]ScanKey=[2024-02-25 : 2024-03-04]ScanKey=[2024-03-05 : 2024-03-13]ScanKey=[2024-03-14 : 2024-03-22]ScanKey=[2024-03-23 : 2024-03-31]ScanKey=[2024-04-01 : 2024-04-09]ScanKey=[2024-04-10 : 2024-04-18]ScanKey=[2024-04-19 : 2024-04-27]ScanKey=[2024-04-28 : 2024-05-06]ScanKey=[2024-05-07 : 2024-05-15]ScanKey=[2024-05-16 : 2024-05-24]ScanKey=[2024-05-25 : 2024-06-02]ScanKey=[2024-06-03 : 2024-06-11]ScanKey=[2024-06-12 : 2024-06-20]ScanKey=[2024-06-21 : 2024-06-29]ScanKey=[2024-06-30 : 2024-07-08]ScanKey=[2024-07-09 : 2024-07-17]ScanKey=[2024-07-18 : 2024-07-26]ScanKey=[2024-07-27 : 2024-07-29]
- TabletIds: [16915]
- UseSpecificThreadToken: False
- AcquireRuntimeFilterTime: 539ns
- AllocateResourceTime: 437.820us
- GetNextTime: 12.257ms
- MaxScannerThreadNum: 1
- MemoryUsage:
- PeakMemoryUsage: 0.00
- NumScanners: 1
- OpenTime: 0ns
- ProcessConjunctTime: 323.132us
- ProjectionTime: 379.667ms
- RowsReturned: 3.243522M (3243522)
- RowsReturnedRate: 276.006794M /sec
- ScanByteRead: 985.94 MB
- ScanRowsRead: 3.243522M (3243522)
- ScannerWorkerWaitTime: 184.385ms
- TabletNum: 1
- TotalReadThroughput: 0
VScanner:
- ReadColumns: [F18, D, C, A, B, F1, F2, F3, F3s, F5, F6
, F7, F8, F9, F10, F11, F12, F13, F14, F15, F16, F17
]
- PerScannerRunningTime: [643.371ms, ]
- PerScannerRowsRead: [3.24M, ]
- PerScannerWaitTime: [184.385ms, ]
- BlockConvertTime: 0ns
- BlockFetchTime: 620.645ms
- MemoryUsage:
- FreeBlocks: 62.55 MB
- QueuedBlocks: 0.00
- NewlyCreateFreeBlocksNum: 51
- ReaderInitTime: 0ns
- RowsDelFiltered: 0
- ScannerBatchWaitTime: 0ns
- ScannerConvertBlockTime: 0ns
- ScannerCpuTime: 592.566ms
- ScannerCtxSchedCount: 252
- ScannerCtxSchedTime: 22.433ms
- ScannerFilterTime: 4.852ms
- ScannerGetBlockTime: 637.781ms
- ScannerInitTime: 70.968us
- ScannerPrefilterTime: 0ns
- ScannerSchedCount: 160
SegmentIterator:
- BitmapIndexFilterTimer: 895ns
- BlockConditionsFilteredTime: 3.983us
- BlockInitSeekCount: 1.013K (1013)
- BlockInitSeekTime: 335.441us
- BlockInitTime: 904.778us
- BlockLoadTime: 621.982ms
- BlocksLoad: 810
- CachedPagesNum: 9.4K (9400)
- CompressedBytesRead: 0.00
- DecompressorTimer: 0ns
- ExprFilterEvalTime: 0ns
- FirstReadSeekCount: 17.776K (17776)
- FirstReadSeekTime: 7.13ms
- FirstReadTime: 613.67ms
- IOTimer: 0ns
- InvertedIndexFilterTime: 1.395us
- InvertedIndexQueryBitmapCopyTime: 0ns
- InvertedIndexQueryBitmapOpTime: 0ns
- InvertedIndexQueryCacheHit: 0
- InvertedIndexQueryCacheMiss: 0
- InvertedIndexQueryTime: 0ns
- InvertedIndexSearcherOpenTime: 0ns
- InvertedIndexSearcherSearchTime: 0ns
- LazyReadSeekCount: 0
- LazyReadSeekTime: 0ns
- LazyReadTime: 0ns
- NumSegmentFiltered: 0
- NumSegmentTotal: 2
- OutputColumnTime: 2.637ms
- OutputIndexResultColumnTimer: 298.821us
- RawRowsRead: 3.243522M (3243522)
- RowsBitmapIndexFiltered: 0
- RowsBloomFilterFiltered: 0
- RowsConditionsFiltered: 0
- RowsDictFiltered: 0
- RowsInvertedIndexFiltered: 0
- RowsKeyRangeFiltered: 319.409K (319409)
- RowsShortCircuitPredFiltered: 0
- RowsShortCircuitPredInput: 0
- RowsStatsFiltered: 0
- RowsVectorPredFiltered: 0
- RowsVectorPredInput: 0
- SecondReadTime: 0ns
- ShortPredEvalTime: 0ns
- TotalPagesNum: 9.4K (9400)
- UncompressedBytesRead: 0.00
- VectorPredEvalTime: 0ns
PipelineContext:(Active: 7s481ms, % non-child: 0.00%)
- PrepareTime: 1.25ms
- StartTime: 1.943us
Pipeline (pipeline id=0):
PipelineTask (index=0):(Active: 170.508ms, % non-child: 0.00%)
- Sink: ExchangeSinkOperator(dst_id=1177)
- OperatorIds(source2root): [StreamingAggSourceOperator(node_id=1174)]
- CoreChangeTimes: 0
- ExecuteTime: 166.769ms
- CloseTime: 3.669ms
- FinalizeTime: 325ns
- GetBlockTime: 87.561ms
- OpenTime: 27.241us
- PrepareTime: 16.877us
- SinkTime: 79.51ms
- GetBlockCounter: 12
- NumBlockedBySinkTimes: 0
- NumBlockedBySrcTimes: 1
- NumBlockedTimes: 2
- NumScheduleTimes: 5
- NumYieldTimes: 1
- Task1BeginExecuteTime: 7s13ms
- Task2EosTime: 7s255ms
- Task3SrcPendingFinishOverTime: 7s255ms
- Task4DstPendingFinishOverTime: 7s261ms
- Task5TotalTime: 7s480ms
- Task6ClosePipelineTime: 7s480ms
- TaskCpuTime: 154.543ms
- WaitBfTime: 0ns
- WaitSinkTime: 0ns
- WaitSourceTime: 6s905ms
- WaitWorkerTime: 374.16ms
Pipeline (pipeline id=1):
PipelineTask (index=1):(Active: 5s226ms, % non-child: 0.00%)
- Sink: StreamingAggSinkOperator(dst_id=1174)
- OperatorIds(source2root): [ScanOperator(node_id=1164)]
- CoreChangeTimes: 2
- ExecuteTime: 5s225ms
- CloseTime: 578.57us
- FinalizeTime: 218ns
- GetBlockTime: 398.180ms
- OpenTime: 457.575us
- PrepareTime: 10.589us
- SinkTime: 4s813ms
- GetBlockCounter: 800
- NumBlockedBySinkTimes: 0
- NumBlockedBySrcTimes: 1
- NumBlockedTimes: 2
- NumScheduleTimes: 51
- NumYieldTimes: 49
- Task1BeginExecuteTime: 136.299ms
- Task2EosTime: 6s968ms
- Task3SrcPendingFinishOverTime: 6s968ms
- Task4DstPendingFinishOverTime: 6s968ms
- Task5TotalTime: 6s969ms
- Task6ClosePipelineTime: 6s969ms
- TaskCpuTime: 4s54ms
- WaitBfTime: 0ns
- WaitSinkTime: 0ns
- WaitSourceTime: 22.674ms
- WaitWorkerTime: 1s720ms
Instance 26d0866efab94717-a8ef244b11cb8f1d (host=TNetworkAddress(hostname:192.168.30.224, port:9060)):
VDataStreamSender (dst_id=1177, dst_fragments=[{"26d0866efab94717-a8ef244b11cb8f1f", "26d0866efab94717-a8ef244b11cb8f20"}]):(Active: 77.958ms, % non-child: 0.00%)
- BlocksSent: 13
- BrpcSendTime: 0ns
- BrpcSendTime.Wait: 0ns
- BytesSent: 4.20 MB
- CompressTime: 26.371ms
- IgnoreRows: 0
- LocalBytesSent: 10.64 MB
- LocalSendTime: 345.231us
- LocalSentRows: 24.395K (24395)
- MemoryUsage:
- PeakMemoryUsage: 24.63 MB
- OverallThroughput: 53.81815242767334 MB/sec
- RpcAvgTime: 2.377ms
- RpcCount: 6
- RpcMaxTime: 14.264ms
- RpcMinTime: 14.264ms
- RpcSumTime: 14.264ms
- SerializeBatchTime: 32.186ms
- SplitBlockDistributeByChannelTime: 42.273ms
- SplitBlockHashComputeTime: 3.195ms
- UncompressedRowBatchSize: 10.57 MB
VAGGREGATION_NODE (id=1174):(Active: 291.455us, % non-child: 0.00%)
- AggInfos: (_is_merge: false, _needs_finalize: false, Streaming Preaggregation: true, agg size: 17, limit: -1)
- BuildConvertToPartitionedTime: 0ns
- DeserializeAndMergeTime: 0ns
- ExecTime: 4s599ms
- ExprTime: 23.495ms
- GetResultsTime: 96.192ms
- HashTableComputeTime: 1s974ms
- HashTableInputCount: 3.25818M (3258180)
- HashTableIterateTime: 717.639us
- HashTableSize: 48.647K (48647)
- InsertKeysToColumnTime: 8.703ms
- MaxRowSizeInBytes: 175
- MaxSizeInBlockQueue: 0.00
- MaxSizeOfBlockQueue: 0
- MemoryUsage:
- HashTable: 1.50 MB
- PeakMemoryUsage: 0.00
- SerializeKeyArena: 70.74 MB
- MergeTime: 0ns
- ProjectionTime: 0ns
- RowsReturned: 48.647K (48647)
- RowsReturnedRate: 166.910843M /sec
- SerializeDataTime: 86.465ms
- SerializeKeyTime: 712.837ms
- SerializeResultTime: 96.354ms
- StreamingAggTime: 0ns
VNewOlapScanNode(MAIN_TABLE) (id=1164):(Active: 24.447ms, % non-child: 0.00%)
- RuntimeFilters: :
- PushDownPredicates: []
- KeyRanges: ScanKeys:ScanKey=[2023-07-06 : 2023-07-14]ScanKey=[2023-07-15 : 2023-07-23]ScanKey=[2023-07-24 : 2023-08-01]ScanKey=[2023-08-02 : 2023-08-10]ScanKey=[2023-08-11 : 2023-08-19]ScanKey=[2023-08-20 : 2023-08-28]ScanKey=[2023-08-29 : 2023-09-06]ScanKey=[2023-09-07 : 2023-09-15]ScanKey=[2023-09-16 : 2023-09-24]ScanKey=[2023-09-25 : 2023-10-03]ScanKey=[2023-10-04 : 2023-10-12]ScanKey=[2023-10-13 : 2023-10-21]ScanKey=[2023-10-22 : 2023-10-30]ScanKey=[2023-10-31 : 2023-11-08]ScanKey=[2023-11-09 : 2023-11-17]ScanKey=[2023-11-18 : 2023-11-26]ScanKey=[2023-11-27 : 2023-12-05]ScanKey=[2023-12-06 : 2023-12-14]ScanKey=[2023-12-15 : 2023-12-23]ScanKey=[2023-12-24 : 2024-01-01]ScanKey=[2024-01-02 : 2024-01-10]ScanKey=[2024-01-11 : 2024-01-19]ScanKey=[2024-01-20 : 2024-01-28]ScanKey=[2024-01-29 : 2024-02-06]ScanKey=[2024-02-07 : 2024-02-15]ScanKey=[2024-02-16 : 2024-02-24]ScanKey=[2024-02-25 : 2024-03-04]ScanKey=[2024-03-05 : 2024-03-13]ScanKey=[2024-03-14 : 2024-03-22]ScanKey=[2024-03-23 : 2024-03-31]ScanKey=[2024-04-01 : 2024-04-09]ScanKey=[2024-04-10 : 2024-04-18]ScanKey=[2024-04-19 : 2024-04-27]ScanKey=[2024-04-28 : 2024-05-06]ScanKey=[2024-05-07 : 2024-05-15]ScanKey=[2024-05-16 : 2024-05-24]ScanKey=[2024-05-25 : 2024-06-02]ScanKey=[2024-06-03 : 2024-06-11]ScanKey=[2024-06-12 : 2024-06-20]ScanKey=[2024-06-21 : 2024-06-29]ScanKey=[2024-06-30 : 2024-07-08]ScanKey=[2024-07-09 : 2024-07-17]ScanKey=[2024-07-18 : 2024-07-26]ScanKey=[2024-07-27 : 2024-07-29]
- TabletIds: [16919]
- UseSpecificThreadToken: False
- AcquireRuntimeFilterTime: 688ns
- AllocateResourceTime: 373.882us
- GetNextTime: 24.823ms
- MaxScannerThreadNum: 1
- MemoryUsage:
- PeakMemoryUsage: 0.00
- NumScanners: 1
- OpenTime: 0ns
- ProcessConjunctTime: 235.501us
- ProjectionTime: 365.144ms
- RowsReturned: 3.25818M (3258180)
- RowsReturnedRate: 133.272511M /sec
- ScanByteRead: 990.59 MB
- ScanRowsRead: 3.25818M (3258180)
- ScannerWorkerWaitTime: 224.437ms
- TabletNum: 1
- TotalReadThroughput: 0
VScanner:
- ReadColumns: [F18, D, C, A, B, F1, F2, F3, F3s, F5, F6
, F7, F8, F9, F10, F11, F12, F13, F14, F15, F16, F17
]
- PerScannerRunningTime: [680.649ms, ]
- PerScannerRowsRead: [3.26M, ]
- PerScannerWaitTime: [224.437ms, ]
- BlockConvertTime: 0ns
- BlockFetchTime: 668.895ms
- MemoryUsage:
- FreeBlocks: 65.01 MB
- QueuedBlocks: 0.00
- NewlyCreateFreeBlocksNum: 53
- ReaderInitTime: 0ns
- RowsDelFiltered: 0
- ScannerBatchWaitTime: 0ns
- ScannerConvertBlockTime: 0ns
- ScannerCpuTime: 621.120ms
- ScannerCtxSchedCount: 232
- ScannerCtxSchedTime: 43.41ms
- ScannerFilterTime: 5.296ms
- ScannerGetBlockTime: 674.591ms
- ScannerInitTime: 95.889us
- ScannerPrefilterTime: 0ns
- ScannerSchedCount: 161
SegmentIterator:
- BitmapIndexFilterTimer: 781ns
- BlockConditionsFilteredTime: 4.163us
- BlockInitSeekCount: 1.065K (1065)
- BlockInitSeekTime: 353.192us
- BlockInitTime: 825.378us
- BlockLoadTime: 670.76ms
- BlocksLoad: 813
- CachedPagesNum: 9.472K (9472)
- CompressedBytesRead: 0.00
- DecompressorTimer: 0ns
- ExprFilterEvalTime: 0ns
- FirstReadSeekCount: 17.864K (17864)
- FirstReadSeekTime: 7.525ms
- FirstReadTime: 660.370ms
- IOTimer: 0ns
- InvertedIndexFilterTime: 1.717us
- InvertedIndexQueryBitmapCopyTime: 0ns
- InvertedIndexQueryBitmapOpTime: 0ns
- InvertedIndexQueryCacheHit: 0
- InvertedIndexQueryCacheMiss: 0
- InvertedIndexQueryTime: 0ns
- InvertedIndexSearcherOpenTime: 0ns
- InvertedIndexSearcherSearchTime: 0ns
- LazyReadSeekCount: 0
- LazyReadSeekTime: 0ns
- LazyReadTime: 0ns
- NumSegmentFiltered: 0
- NumSegmentTotal: 1
- OutputColumnTime: 1.720ms
- OutputIndexResultColumnTimer: 346.917us
- RawRowsRead: 3.25818M (3258180)
- RowsBitmapIndexFiltered: 0
- RowsBloomFilterFiltered: 0
- RowsConditionsFiltered: 0
- RowsDictFiltered: 0
- RowsInvertedIndexFiltered: 0
- RowsKeyRangeFiltered: 357.674K (357674)
- RowsShortCircuitPredFiltered: 0
- RowsShortCircuitPredInput: 0
- RowsStatsFiltered: 0
- RowsVectorPredFiltered: 0
- RowsVectorPredInput: 0
- SecondReadTime: 0ns
- ShortPredEvalTime: 0ns
- TotalPagesNum: 9.472K (9472)
- UncompressedBytesRead: 0.00
- VectorPredEvalTime: 0ns
PipelineContext:(Active: 18s221ms, % non-child: 0.00%)
- PrepareTime: 809.783us
- StartTime: 1.935us
Pipeline (pipeline id=0):
PipelineTask (index=0):(Active: 177.913ms, % non-child: 0.00%)
- Sink: ExchangeSinkOperator(dst_id=1177)
- OperatorIds(source2root): [StreamingAggSourceOperator(node_id=1174)]
- CoreChangeTimes: 1
- ExecuteTime: 174.600ms
- CloseTime: 3.239ms
- FinalizeTime: 559ns
- GetBlockTime: 96.421ms
- OpenTime: 30.320us
- PrepareTime: 13.13us
- SinkTime: 78.32ms
- GetBlockCounter: 12
- NumBlockedBySinkTimes: 0
- NumBlockedBySrcTimes: 1
- NumBlockedTimes: 2
- NumScheduleTimes: 4
- NumYieldTimes: 1
- Task1BeginExecuteTime: 18s20ms
- Task2EosTime: 18s194ms
- Task3SrcPendingFinishOverTime: 18s194ms
- Task4DstPendingFinishOverTime: 18s194ms
- Task5TotalTime: 18s221ms
- Task6ClosePipelineTime: 18s221ms
- TaskCpuTime: 174.307ms
- WaitBfTime: 0ns
- WaitSinkTime: 0ns
- WaitSourceTime: 17s872ms
- WaitWorkerTime: 147.637ms
Pipeline (pipeline id=1):
PipelineTask (index=1):(Active: 4s916ms, % non-child: 0.00%)
- Sink: StreamingAggSinkOperator(dst_id=1174)
- OperatorIds(source2root): [ScanOperator(node_id=1164)]
- CoreChangeTimes: 2
- ExecuteTime: 4s914ms
- CloseTime: 645.373us
- FinalizeTime: 391ns
- GetBlockTime: 396.270ms
- OpenTime: 392.483us
- PrepareTime: 10.45us
- SinkTime: 4s506ms
- GetBlockCounter: 804
- NumBlockedBySinkTimes: 0
- NumBlockedBySrcTimes: 1
- NumBlockedTimes: 2
- NumScheduleTimes: 49
- NumYieldTimes: 47
- Task1BeginExecuteTime: 182.341ms
- Task2EosTime: 18s19ms
- Task3SrcPendingFinishOverTime: 18s19ms
- Task4DstPendingFinishOverTime: 18s19ms
- Task5TotalTime: 18s20ms
- Task6ClosePipelineTime: 18s20ms
- TaskCpuTime: 4s287ms
- WaitBfTime: 0ns
- WaitSinkTime: 0ns
- WaitSourceTime: 10.597ms
- WaitWorkerTime: 13s93ms
LoadChannels:
