2.1.4版本 python使用arrow flight sql ,连接不会断开 ,连接数占满,达到100

Viewed 252
2.1

image.png
conn = flight_sql.connect(uri="grpc://192.168.6.11:9040", db_kwargs={
adbc_driver_manager.DatabaseOptions.USERNAME.value: "root",
adbc_driver_manager.DatabaseOptions.PASSWORD.value: "yutu@611Security"
})

创建游标

cur = conn.cursor()
cur.close()
conn.close()
有close 为啥不管用

郭明昊9
3 Answers
徐振超@SelectDB6266

这个问题是因为没有限制单个用户的 arrow flight 连接数小于 user property 的 max_user_connections，默认100。

  1. 可以在 fe.conf 中增加 arrow_flight_token_cache_size=50 来规避这个问题。arrow_flight_token_cache_size的含义见下面。
  2. 或者调大当前用户的 max_user_connections 到 1000

目前 arrow flight 连接默认 3天 超时断开，目前只强制连接数小于 qe_max_connection/2，超过时依据lru淘汰，qe_max_connection是fe所有用户的总连接数，默认1024。具体可以看下面两个 fe config 的描述

@ConfField(description = {"Arrow Flight Server中所有用户token的缓存上限，超过后LRU淘汰，默认值为512, "
            + "并强制限制小于 qe_max_connection/2, 避免`Reach limit of connections`, "
            + "因为arrow flight sql是无状态的协议，连接通常不会主动断开，"
            + "bearer token 从 cache 淘汰的同时会 unregister Connection.",
            "The cache limit of all user tokens in Arrow Flight Server. which will be eliminated by"
            + "LRU rules after exceeding the limit, the default value is 512, the mandatory limit is "
            + "less than qe_max_connection/2 to avoid `Reach limit of connections`, "
            + "because arrow flight sql is a stateless protocol, the connection is usually not actively "
            + "disconnected, bearer token is evict from the cache will unregister ConnectContext."})
    public static int arrow_flight_token_cache_size = 512;

    @ConfField(description = {"Arrow Flight Server中用户token的存活时间，自上次写入后过期时间，单位分钟，默认值为4320，即3天",
            "The alive time of the user token in Arrow Flight Server, expire after write, unit minutes,"
            + "the default value is 4320, which is 3 days"})
    public static int arrow_flight_token_alive_time = 4320;

后面会修复这个问题，预期这一两个月抽空

xinyiZzz362