【已解决】doris2.1.3版本，flink-doris-connector方式部分字段更新问题

Viewed 83
2.1 ingestion

sink端配置

	Properties properties = new Properties();
        // 上游是 json 写入时，需要开启配置
        properties.setProperty("format", "json");
        properties.setProperty("read_json_by_line", "true");
        properties.setProperty("sink.enable-2pc", "true");
        properties.setProperty("sink.properties.columns", "id,name,age,op_time,__DORIS_DELETE_SIGN__");
        properties.setProperty("sink.properties.partial.columns", "true");

	DorisExecutionOptions.Builder executionBuilder = DorisExecutionOptions.builder();
        executionBuilder
                .setLabelPrefix("label-doris" + System.currentTimeMillis())
                .setDeletable(true)
//                .setBufferSize(8 * 1024)
//                .setBufferCount(3)
//                .setDeletable(false)
                .setStreamLoadProp(properties);

在这个配置下，传入json数据想到达部分字段更新的效果

{
  "type":"insert",
  "ts":1662560666043,
  "data":[
    {
      "id":"2",
      "name":"anni"
    }
  ]
}

目前不能支持部分列更新
请问各位大佬，是哪里出现了问题？

xiyang.zdz5
2 Answers

改成properties.setProperty("partial_columns", "true");

vinlee218

datastream不需要sink.properties前缀
properties.setProperty("columns", "id,name,age,op_time,DORIS_DELETE_SIGN");
properties.setProperty("partial_columns", "true");

JNSimba121