3.0.3 请教,Doris在单表出现查询效率低的问题,请大神帮忙分析一下

Viewed 23

k8s部署,6个be [24C,64g],5个fe[8C,32g]
存算分离
版本:3.0.3

建表语句:

CREATE TABLE `ods_platform_order_center_poc_order_item` (
  `dt` date NOT NULL,
  `order_item_id` bigint NOT NULL COMMENT '订单明细id',
  `order_id` bigint NOT NULL COMMENT '订单id',
  `id` bigint NOT NULL COMMENT '主键',
  `biz_channel` int NOT NULL COMMENT '业务渠道',
  `biz_order_id` varchar(32) NOT NULL COMMENT '业务订单编号',
  `buyer_unit_id` bigint NOT NULL COMMENT '买家业务单元id',
  `seller_id` bigint NOT NULL COMMENT '卖家id',
  `seller_shop_id` bigint NOT NULL COMMENT '卖家门店id',
  `goods_snapshot_id` bigint NOT NULL COMMENT '商品快照id',
  `goods_type` tinyint NOT NULL COMMENT '商品类型',
  `sku_id` bigint NOT NULL COMMENT 'skuid',
  `order_num` decimal(10, 2) NOT NULL COMMENT '下单的sku数量',
  `price_num_of_sku` decimal(10, 2) NOT NULL DEFAULT "0.00" COMMENT 'SKU含计价单位数量',
  `raw_unit_price` decimal(18, 9) NOT NULL COMMENT '原计价单位单价',
  `unit_price` decimal(18, 9) NOT NULL COMMENT '下单时计价单位单价',
  `goods_discount` decimal(10, 2) NOT NULL DEFAULT "0.00" COMMENT '履约前折合到该OI的优惠总金额',
  `parent_order_item_id` bigint NOT NULL COMMENT '根订单orderitemid',
  `unit_discount_price` decimal(18, 9) NOT NULL DEFAULT "0.000000000" COMMENT '优惠后计价单位单价',
  `dispatch_num` decimal(10, 2) NOT NULL DEFAULT "0.00" COMMENT '发运量',
  `dispatch_price_num` decimal(10, 2) NOT NULL DEFAULT "0.00" COMMENT '发运计价量',
  `receivable_amount` decimal(10, 2) NOT NULL DEFAULT "0.00" COMMENT '应收金额',
  `performed_discount_amount` decimal(10, 2) NOT NULL DEFAULT "0.00" COMMENT '履约产生的追加优惠',
  `performed_goods_amount` decimal(10, 2) NOT NULL DEFAULT "0.00" COMMENT '按下单单价*履约量计算出的货物价值',
  `performed_price_num` decimal(10, 2) NOT NULL DEFAULT "0.00" COMMENT '履约货物计价单位量',
  `performed_num` decimal(10, 2) NOT NULL DEFAULT "0.00" COMMENT '履约货物sku量',
  `virtual_warehouse_id` bigint NOT NULL DEFAULT "0" COMMENT '虚拟仓库ID',
  `seller_warehouse_code` bigint NOT NULL DEFAULT "0" COMMENT '卖家仓库编码',
  `ct` bigint NOT NULL COMMENT '记录创建时间',
  `ut` bigint NOT NULL COMMENT '记录更新时间',
  `deleted` tinyint NOT NULL DEFAULT "0" COMMENT '删除标识',
  INDEX idx_order_id (`order_id`) USING INVERTED
) ENGINE = OLAP UNIQUE KEY(`dt`, `order_item_id`) AUTO PARTITION BY RANGE (date_trunc(`dt`, 'day')) () 
DISTRIBUTED BY HASH(`order_item_id`) BUCKETS 1 
PROPERTIES ( 
"file_cache_ttl_seconds" = "0", 
"is_being_synced" = "false",
 "storage_medium" = "hdd", 
 "storage_format" = "V2",
 "inverted_index_storage_format" = "V2", 
 "enable_unique_key_merge_on_write" = "true",
 "light_schema_change" = "true", 
 "storage_vault_name" = "s3_vault", 
 "disable_auto_compaction" = "false",
 "enable_single_replica_compaction" = "false", 
 "group_commit_interval_ms" = "10000", 
 "group_commit_data_bytes" = "134217728",
 "enable_mow_light_delete" = "false" 
  );

image.png
按照dt分区,由于每天大概有200W条数据,大小在100M左右,参考官方文档分桶的的建议,为了防止元数据管理压力,bucket设置为1。

偶尔会出现单表查询1s+

select * from ods.ods_platform_order_center_poc_order_item where   dt > '2024-08-01' and  dt < '2024-08-30' and  order_id in (
2412625406, 2412625512, 2412625513, 2412625619, 2412625694, 2412625698, 2412625805, 2412625814, 2412625823, 2412625839, 2412625879, 2412625930, 2412626038, 2412626051, 2412626066, 2412626082, 2412626132, 2412626161, 2412626209, 2412626260, 2412626277, 2412626319, 2412626328, 2412626334, 2412626335, 2412626363, 2412626393, 2412626410, 2412626417, 2412626520, 2412626540, 2412626856, 2412627571, 2412627660, 2412627667, 2412627670, 2412627682, 2412627711, 2412627720, 2412627827, 2412627841, 2412627852, 2412627911, 2412628058, 2412628174, 2412628204, 2412628223, 2412628224, 2412628242, 2412628260, 2412628399, 2412628595, 2412628662, 2412628708, 2412628723, 2412628748, 2412628749, 2412628796, 2412628861, 2412629088, 2412629193, 2412629253, 2412629280, 2412629318, 2412629383, 2412629413, 2412629430, 2412629597, 2412629775, 2412629783, 2412629967, 2412629968, 2412630146, 2412630180, 2412630238, 2412630283, 2412630325, 2412630355, 2412630364, 2412630370, 2412630461, 2412630557, 2412630558, 2412630648, 2412630664, 2412630679, 2412630681, 2412630693, 2412630706, 2412630723, 2412630733, 2412630744, 2412630782, 2412630794, 2412630797, 2412630814, 2412630844, 2412630860, 2412630883, 2412630888, 2412630940, 2412631048, 2412631076, 2412631142, 2412631164, 2412631181, 2412631279, 2412631323, 2412631345, 2412631478, 2412631479, 2412631481, 2412631484, 2412631491, 2412631492, 2412631568, 2412631622, 2412631632, 2412631635, 2412631649, 2412631652, 2412631726, 2412631739, 2412631746, 2412631748, 2412631761, 2412631785, 2412631795, 2412631832, 2412631910, 2412631958, 2412631983, 2412631997, 2412632033, 2412632073, 2412632075, 2412632138, 2412632150, 2412632162, 2412632170, 2412632171, 2412632172, 2412632213, 2412632222, 2412632234, 2412632308, 2412632315, 2412632322, 2412632350, 2412632370, 2412632398, 2412632399, 2412632428, 2412632445, 2412632447, 2412632461, 2412632479, 2412632487, 2412632493, 2412632499, 2412632513, 2412632514, 2412632523, 2412632547, 2412632567, 2412632582, 2412632601, 2412632636, 2412632640, 2412632658, 2412632676, 2412632691, 2412632700, 2412632716, 2412632717, 2412632720, 2412632743, 2412632791, 2412632922, 2412632937, 2412632949, 2412633020, 2412633034, 2412633050, 2412633077, 2412633127, 2412633130, 2412633144, 2412633160, 2412633237, 2412633247, 2412633249, 2412633250, 2412633281, 2412633286, 2412633314, 2412633326, 2412633353, 2412633368, 2412633449, 2412633454, 2412633455, 2412633459, 2412633467, 2412633562, 2412633591, 2412633640, 2412633661, 2412633673, 2412633678, 2412633690, 2412633710, 2412633728, 2412633741, 2412633742, 2412633758, 2412633760, 2412633774, 2412633785, 2412633787, 2412633826, 2412633876, 2412633877, 2412633883, 2412633888, 2412633889, 2412633913, 2412633922, 2412633938, 2412633975, 2412633985, 2412633989, 2412634017, 2412634091, 2412634157, 2412634173, 2412634179, 2412634210, 2412634212, 2412634220, 2412634232, 2412634271, 2412634294, 2412634304, 2412634317, 2412634395, 2412634404, 2412634522, 2412634554, 2412634565, 2412634585, 2412634604, 2412634621, 2412634639)

已经设置了 max_scan_key_num 为1024
image.png

profile 文件太长了,贴不上, 麻烦➕vx:jinxiangWang0522,谢谢

1 Answers

DISTRIBUTED BY HASH(order_item_id) BUCKETS 1

bucket 给多些试试