【已解决】arrow flight port 监听到ipv6上导致无法访问

Viewed 65

doris版本: 2.1.5
arrow jdbc driver 版本: 17.0.0.0 以及 16.0.0.0

fe 信息如下:
image.png

fe配置文件:
image.png

本地执行 netstat -nltp | grep 8070 , 得到以下结果:
image.png

监听的8070端口是tcp6协议而不是tcp协议,client无法登录。
当前设备并没有ipv6地址:
image.png

client报错信息如下:
image.png

Oct 16, 2024 7:40:39 PM org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.memory.BaseAllocator <clinit>
INFO: Debug mode disabled. Enable with the VM option -Darrow.memory.debug.allocator=true.
Oct 16, 2024 7:40:39 PM org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.memory.DefaultAllocationManagerOption getDefaultAllocationManagerFactory
INFO: allocation manager type not specified, using netty as the default type
Oct 16, 2024 7:40:39 PM org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.memory.CheckAllocator reportResult
INFO: Using DefaultAllocationManager at memory/netty/DefaultAllocationManagerFactory.class
Oct 16, 2024 7:40:39 PM org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.flight.auth2.ClientHandshakeWrapper doClientHandshake
SEVERE: Failed on completing future
org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.flight.FlightRuntimeException: UNAVAILABLE: io exception
        at org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.flight.CallStatus.toRuntimeException(CallStatus.java:121)
        at org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.flight.grpc.StatusUtils.fromGrpcRuntimeException(StatusUtils.java:161)
        at org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.flight.grpc.StatusUtils.fromThrowable(StatusUtils.java:182)
        at org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.flight.auth2.ClientHandshakeWrapper.doClientHandshake(ClientHandshakeWrapper.java:55)
        at org.apache.arrow.driver.jdbc.shaded.org.apache.arrow.flight.FlightClient.handshake(FlightClient.java:212)
        at org.apache.arrow.driver.jdbc.client.utils.ClientAuthenticationUtils.getAuthenticate(ClientAuthenticationUtils.java:107)
        at org.apache.arrow.driver.jdbc.client.utils.ClientAuthenticationUtils.getAuthenticate(ClientAuthenticationUtils.java:92)
        at org.apache.arrow.driver.jdbc.client.ArrowFlightSqlClientHandler$Builder.build(ArrowFlightSqlClientHandler.java:897)
        at org.apache.arrow.driver.jdbc.ArrowFlightConnection.createNewClientHandler(ArrowFlightConnection.java:116)
        at org.apache.arrow.driver.jdbc.ArrowFlightConnection.createNewConnection(ArrowFlightConnection.java:88)
        at org.apache.arrow.driver.jdbc.ArrowFlightJdbcDriver.connect(ArrowFlightJdbcDriver.java:90)
        at org.apache.arrow.driver.jdbc.ArrowFlightJdbcDriver.connect(ArrowFlightJdbcDriver.java:46)
        at java.sql.DriverManager.getConnection(DriverManager.java:664)
        at java.sql.DriverManager.getConnection(DriverManager.java:247)
        at org.example.App.main(App.java:24)
Caused by: org.apache.arrow.driver.jdbc.shaded.io.netty.channel.AbstractChannel$AnnotatedNoRouteToHostException: No route to host: DorisFe0001/172.16.10.32:8070
Caused by: java.net.NoRouteToHostException: No route to host
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716)
        at org.apache.arrow.driver.jdbc.shaded.io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:336)
        at org.apache.arrow.driver.jdbc.shaded.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:339)
        at org.apache.arrow.driver.jdbc.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776)
        at org.apache.arrow.driver.jdbc.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
        at org.apache.arrow.driver.jdbc.shaded.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
        at org.apache.arrow.driver.jdbc.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
        at org.apache.arrow.driver.jdbc.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:994)
        at org.apache.arrow.driver.jdbc.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at org.apache.arrow.driver.jdbc.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:748)

client 代码:

public static void main(String[] args) throws Exception{
        Class.forName("org.apache.arrow.driver.jdbc.ArrowFlightJdbcDriver");
        String DB_URL = "jdbc:arrow-flight-sql://172.16.10.32:8070?useServerPrepStmts=false"
                + "&cachePrepStmts=true&useSSL=false&useEncryption=false";
        String USER = "root";
        String PASS = "root123.";

        Connection conn = DriverManager.getConnection(DB_URL, USER, PASS);
        Statement stmt = conn.createStatement();
        ResultSet resultSet = stmt.executeQuery("select * from t1;");
        while (resultSet.next()) {
            System.out.println(resultSet.toString());
        }

        resultSet.close();
        stmt.close();
        conn.close();
    }

最新补充情况,centos系统禁用ipv6后,正常监听 0.0.0.0::8070, 但是client还是无法连接。参考下图:
image.png

有同学认为是我的client与fe节点的网络不通。这里补充详细的信息:

  1. client所在设备正常连接到fe的9030端口,即jdbc协议;也可以连接到be节点的8040,8060端口。
  2. 将client包放在fe节点执行,依然报同样的错。参考下图:
    image.png
  3. 在fe节点telnet 本机ip的8070端口是没问题的:
    image.png
  4. 在非doris集群节点(网络可达),无法telnet通 fe节点的8070端口:
    image.png
3 Answers

查看fe的日志,显示flight service已经启动:
image.png

fe其他关于arrow flight日志如下:
image.png

另外BE也是一样的问题,监听的8070 端口是tcp6 协议。

显示监听tcp6,tcp也是可以用的,传输层tcp协议和网络层IPv4/IPv6是独立的,并不能说明当前进程监听的是IPv6,实际还是监听的IPv4,这连不上和tcp6没关系。

大概率是你 client 所在的机器,连不上 FE 的 172.16.10.32 这个内网地址,

  1. mysql -h172.16.10.32 -P9030 看 mysql client 能否连上
  2. 发下 show frontends; 看下 fe 的 ip 是多少