stream load error: HTTP/1.1 307 Temporary Redirect

Viewed 53

使用 flink-doris-connector 经常报错 org.apache.doris.flink.exception.StreamLoadException: stream load error: HTTP/1.1 307 Temporary Redirect ,白天晚上都有报错,看着跟数据量没关系。fe 日志没打印相关错误日志
doris 版本:2.1.7
connector 版本:flink-doris-connector-1.17-1.6.1.jar
报错堆栈:

java.lang.Exception: Could not perform checkpoint 2204 for operator Source: kafka_doris_mysql_order[133] -> Calc[134] -> kafka_doris_mysql_order[135]: Writer -> kafka_doris_mysql_order[135]: Committer (1/1)#0.
	at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointAsyncInMailbox(StreamTask.java:1184) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$triggerCheckpointAsync$13(StreamTask.java:1131) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail(MailboxProcessor.java:398) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsWhenDefaultActionUnavailable(MailboxProcessor.java:367) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:352) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:229) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:839) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:788) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:931) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) ~[flink-dist-1.17.2.jar:1.17.2]
	at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_192]
Caused by: org.apache.doris.flink.exception.DorisRuntimeException: org.apache.doris.flink.exception.StreamLoadException: stream load error: HTTP/1.1 307 Temporary Redirect
	at org.apache.doris.flink.sink.writer.DorisStreamLoad.stopLoad(DorisStreamLoad.java:263) ~[flink-doris-connector-1.17-1.6.1.jar:1.6.1]
	at org.apache.doris.flink.sink.writer.DorisWriter.prepareCommit(DorisWriter.java:249) ~[flink-doris-connector-1.17-1.6.1.jar:1.6.1]
	at org.apache.flink.streaming.runtime.operators.sink.SinkWriterOperator.emitCommittables(SinkWriterOperator.java:196) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.operators.sink.SinkWriterOperator.prepareSnapshotPreBarrier(SinkWriterOperator.java:166) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.prepareSnapshotPreBarrier(RegularOperatorChain.java:89) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointState(SubtaskCheckpointCoordinatorImpl.java:321) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$14(StreamTask.java:1299) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:1287) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointAsyncInMailbox(StreamTask.java:1172) ~[flink-dist-1.17.2.jar:1.17.2]
	... 14 more
Caused by: org.apache.doris.flink.exception.StreamLoadException: stream load error: HTTP/1.1 307 Temporary Redirect
	at org.apache.doris.flink.sink.writer.DorisStreamLoad.handlePreCommitResponse(DorisStreamLoad.java:253) ~[flink-doris-connector-1.17-1.6.1.jar:1.6.1]
	at org.apache.doris.flink.sink.writer.DorisStreamLoad.stopLoad(DorisStreamLoad.java:261) ~[flink-doris-connector-1.17-1.6.1.jar:1.6.1]
	at org.apache.doris.flink.sink.writer.DorisWriter.prepareCommit(DorisWriter.java:249) ~[flink-doris-connector-1.17-1.6.1.jar:1.6.1]
	at org.apache.flink.streaming.runtime.operators.sink.SinkWriterOperator.emitCommittables(SinkWriterOperator.java:196) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.operators.sink.SinkWriterOperator.prepareSnapshotPreBarrier(SinkWriterOperator.java:166) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.prepareSnapshotPreBarrier(RegularOperatorChain.java:89) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointState(SubtaskCheckpointCoordinatorImpl.java:321) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$14(StreamTask.java:1299) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:1287) ~[flink-dist-1.17.2.jar:1.17.2]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointAsyncInMailbox(StreamTask.java:1172) ~[flink-dist-1.17.2.jar:1.17.2]
	... 14 more
2 Answers

这是 307 重定向的问题,BE 地址和 flink 环境telnet ip port 是正常通讯吗

估计存在部分BE节点和 flink 环境telnet ip port 是不正常通讯