RoutineLoad导入之JSON格式配置问题

Viewed 37
2.0

以下是canal写入kafka后的json格式，根路径是{}，而data节点是个数组，我是否能通过配置实现对该json的导入？

主要是 json_root参数和strip_outer_array参数的优先级文档中没有提及。

求大佬们解答

{
    "data": [
        {
            "config_id": "1",
            "config_name": "测试oa回调1",
        },
        {
            "config_id": "2",
            "config_name": "测试回调2",
        }
    ],
    "database": "db",
    "es": 1724210500000,
    "id": 13471,
    "isDdl": false,
    "mysqlType": {
        "config_id": "bigint(20)",
        "config_name": "varchar(50)"
    },
    "old": [
        {
            "last_update_time": "2024-08-15 19:10:03",
            "data_flag": "1"
        },
        {
            "last_update_time": "2023-09-11 15:16:39",
            "data_flag": "1"
        }
    ],
    "pkNames": [
        "config_id"
    ],
    "sql": "",
    "sqlType": {
        "config_id": -5,
        "config_name": 12
    },
    "table": "db_table",
    "ts": 1724210500349,
    "type": "UPDATE"
}

如果不行，是否要自己写逻辑进行格式转换，拆分成两条消息再写入kafka？

KKWong3
1 Answers

先配置了json_root=$.data，提示：
Reason: JSON data is array-object, strip_outer_array must be TRUE.. src line [{"data":[{"cus_id":"1000000","alloc_source":"0","follow_bd_id":"0","...

然后追加配置了strip_outer_array=true，提示：
Reason: [NOT_FOUND]Not found target filed, err: The JSON field referenced does not exist in this object., msg: unable to find field: data. src line [];

感觉是有顺序的，先是处理了strip_outer_array，然后才是json_root

KKWong3