从hive表中导入数据到doris,如何保证任务稳定导入?

Viewed 51

Doris版本2.1.5
问题:
1、从hive中导入数据到doris,报错内存不足,如何保证在内存有限的情况下,导入成功?
2、是否可以设置导入时任务使用的内存大小?
导入SQL如下:

insert into ad_monitor.huan_ad_monitor_impression_log
select hk,
       hp,
       platform,
       hUpAdId,
       hAdId,
       motype,
       third,
       logDate,
       logTime,
       serialNum,
       hm,
       mac,
       wifimac,
       hdnum,
       hmac,
       realIp,
       ip,
       `time`,
       nc,
       sign,
       signver,
       signCorrect,
       hmzk,
       hmzp,
       hmzsalt,
       provinceName,
       provinceCode,
       cityName,
       cityCode,
       countyName,
       countyCode,
       ua,
       hpid,
       hpsid,
       dt,
       tr,
       hCreativeId,
       hGroupId,
       hPlayId,
       hVideoId,
       source,
       hbd,
       ipipProvinceName,
       ipipCityName,
       ipipCountyName,
       ipipCountyCode,
       hChip,
       appVersion,
       sdkVersion,
       idfa,
       idfaMd5,
       imeiMd5,
       oaid,
       oaidMd5,
       hprid,
       hadvid,
       os,
       hds,
       hprice
from hive.ad_monitor.huan_ad_monitor_impression_log_orc
where dt = '2024-08-13';

报错信息
image.png

1 Answers

系统本身内存看着比较小,推荐拆分导入