Doris-3.0.1版本在k8s环境使用FQDN,找不到master

Viewed 132

RuntimeLogger 2024-09-09 09:07:40,957 INFO (main|1) [LogUtils.stdout():50] StdoutLogger 2024-09-09 09:07:40,955 Build version: doris-3.0.1-rc04-patch1
RuntimeLogger 2024-09-09 09:07:40,961 INFO (main|1) [LogUtils.stdout():50] StdoutLogger 2024-09-09 09:07:40,961 Build time: Sun, 25 Aug 2024 20:45:41 CST
RuntimeLogger 2024-09-09 09:07:40,962 INFO (main|1) [LogUtils.stdout():50] StdoutLogger 2024-09-09 09:07:40,962 Build info: vm-209
RuntimeLogger 2024-09-09 09:07:40,962 INFO (main|1) [LogUtils.stdout():50] StdoutLogger 2024-09-09 09:07:40,962 Build hash: git://vm-209@8c4e5730a1456cbb6d5f82be49686bed8696e852
RuntimeLogger 2024-09-09 09:07:40,963 INFO (main|1) [LogUtils.stdout():50] StdoutLogger 2024-09-09 09:07:40,962 Java compile version: openjdk full version "17.0.2+8-86"
RuntimeLogger 2024-09-09 09:07:40,963 INFO (main|1) [DorisFE.printVersion():401] Build version: doris-3.0.1-rc04-patch1
RuntimeLogger 2024-09-09 09:07:40,963 INFO (main|1) [DorisFE.printVersion():402] Build time: Sun, 25 Aug 2024 20:45:41 CST
RuntimeLogger 2024-09-09 09:07:40,963 INFO (main|1) [DorisFE.printVersion():403] Build info: vm-209
RuntimeLogger 2024-09-09 09:07:40,963 INFO (main|1) [DorisFE.printVersion():404] Build hash: git://vm-209@8c4e5730a1456cbb6d5f82be49686bed8696e852
RuntimeLogger 2024-09-09 09:07:40,964 INFO (main|1) [DorisFE.printVersion():405] Java compile version: openjdk full version "17.0.2+8-86"
RuntimeLogger 2024-09-09 09:07:40,964 INFO (main|1) [LogUtils.stdout():50] StdoutLogger 2024-09-09 09:07:40,964 Run FE in the local mode
RuntimeLogger 2024-09-09 09:07:40,969 INFO (main|1) [DorisFE.start():156] Doris FE starting...
RuntimeLogger 2024-09-09 09:07:40,990 INFO (main|1) [FrontendOptions.initAddrUsingFqdn():168] Use FQDN init local addr, FQDN: doriscluster-sample-storageclass1-fe-1.doriscluster-sample-storageclass1-fe-internal.doris.svc.cluster.local, IP: 10.224.137.103
RuntimeLogger 2024-09-09 09:07:41,161 ERROR (main|1) [LogLog.warn():157] log4j:WARN No appenders could be found for logger (io.netty.util.internal.InternalThreadLocalMap).
RuntimeLogger 2024-09-09 09:07:41,162 ERROR (main|1) [LogLog.warn():157] log4j:WARN Please initialize the log4j system properly.
RuntimeLogger 2024-09-09 09:07:41,162 ERROR (main|1) [LogLog.warn():157] log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
RuntimeLogger 2024-09-09 09:07:41,188 INFO (main|1) [ConsistencyChecker.initWorkTime():105] consistency checker will work from 23:00 to 23:00
RuntimeLogger 2024-09-09 09:07:41,290 ERROR (main|1) [Util.report():128] SLF4J: Class path contains multiple SLF4J bindings.
RuntimeLogger 2024-09-09 09:07:41,291 ERROR (main|1) [Util.report():128] SLF4J: Found binding in [jar:file:/opt/apache-doris/fe/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
RuntimeLogger 2024-09-09 09:07:41,291 ERROR (main|1) [Util.report():128] SLF4J: Found binding in [jar:file:/opt/apache-doris/fe/lib/log4j-slf4j-impl-2.18.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
RuntimeLogger 2024-09-09 09:07:41,292 ERROR (main|1) [Util.report():128] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
RuntimeLogger 2024-09-09 09:07:41,294 ERROR (main|1) [Util.report():128] SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory]
RuntimeLogger 2024-09-09 09:07:41,862 INFO (main|1) [PrivTable.addEntry():89] add priv entry: Node_priv,Admin_priv
RuntimeLogger 2024-09-09 09:07:41,862 INFO (main|1) [PrivTable.addEntry():89] add priv entry: Admin_priv
RuntimeLogger 2024-09-09 09:07:41,874 INFO (main|1) [PrivTable.addEntry():89] add priv entry: database privilege.ctl: internal, db: information_schema, priv: Select_priv
RuntimeLogger 2024-09-09 09:07:41,874 INFO (main|1) [PrivTable.addEntry():89] add priv entry: database privilege.ctl: internal, db: mysql, priv: Select_priv
RuntimeLogger 2024-09-09 09:07:41,875 INFO (main|1) [PrivTable.addEntry():89] add priv entry: origWorkloadGroup:normal, priv:Usage_priv
RuntimeLogger 2024-09-09 09:07:41,875 INFO (main|1) [Auth.createUserInternal():546] finished to create user: 'root'@'%', is replay: true
RuntimeLogger 2024-09-09 09:07:41,875 INFO (main|1) [PrivTable.addEntry():89] add priv entry: database privilege.ctl: internal, db: information_schema, priv: Select_priv
RuntimeLogger 2024-09-09 09:07:41,876 INFO (main|1) [PrivTable.addEntry():89] add priv entry: database privilege.ctl: internal, db: mysql, priv: Select_priv
RuntimeLogger 2024-09-09 09:07:41,876 INFO (main|1) [PrivTable.addEntry():89] add priv entry: origWorkloadGroup:normal, priv:Usage_priv
RuntimeLogger 2024-09-09 09:07:41,876 INFO (main|1) [Auth.createUserInternal():546] finished to create user: 'admin'@'%', is replay: true
RuntimeLogger 2024-09-09 09:07:41,878 INFO (main|1) [AuthenticatorManager.():42] authenticate type: DEFAULT
RuntimeLogger 2024-09-09 09:07:41,974 INFO (main|1) [MTMVService.registerHook():66] registerHook: MTMVJobManager
RuntimeLogger 2024-09-09 09:07:41,975 INFO (main|1) [MTMVService.registerHook():66] registerHook: MTMVRelationManager
RuntimeLogger 2024-09-09 09:07:41,987 INFO (main|1) [Env.getSelfHostPort():1407] get self node: HostInfo{host='doriscluster-sample-storageclass1-fe-1.doriscluster-sample-storageclass1-fe-internal.doris.svc.cluster.local', port=9010}
RuntimeLogger 2024-09-09 09:07:41,987 INFO (main|1) [Env.getHelperNodes():1461] get helper nodes: [HostInfo{host='doriscluster-sample-storageclass1-fe-1.doriscluster-sample-storageclass1-fe-internal.doris.svc.cluster.local', port=9010}]
RuntimeLogger 2024-09-09 09:07:41,995 INFO (main|1) [Env.getClusterIdAndRole():1332] finished to get cluster id: 719593487, isElectable: true, role: FOLLOWER and node name: fe_7ce90468_81d2_4dd0_a8e0_b4afca2ae898
RuntimeLogger 2024-09-09 09:07:42,004 INFO (main|1) [Env.loadImage():2037] image does not exist: /opt/apache-doris/fe/doris-meta/image/image.0
RuntimeLogger 2024-09-09 09:07:42,444 WARN (MASTER fe_7ce90468_81d2_4dd0_a8e0_b4afca2ae898(1)|1) [Env.notifyNewFETypeTransfer():2761] notify new FE type transfer: MASTER
RuntimeLogger 2024-09-09 09:07:42,444 INFO (MASTER fe_7ce90468_81d2_4dd0_a8e0_b4afca2ae898(1)|1) [LogUtils.stdout():50] StdoutLogger 2024-09-09 09:07:42,444 notify new FE type transfer: MASTER
RuntimeLogger 2024-09-09 09:07:42,451 INFO (MASTER fe_7ce90468_81d2_4dd0_a8e0_b4afca2ae898(1)|1) [BDBJEJournal.open():532] the very first time to open bdb, dbname is 1
RuntimeLogger 2024-09-09 09:07:42,458 INFO (stateListener|82) [Env$5.runOneCycle():2784] begin to transfer FE type from INIT to MASTER
RuntimeLogger 2024-09-09 09:07:42,460 INFO (stateListener|82) [BDBHA.fencing():79] start fencing, epoch number is 1
RuntimeLogger 2024-09-09 09:07:42,464 INFO (stateListener|82) [Env.transferToMaster():1526] finish replay in 1 msec
RuntimeLogger 2024-09-09 09:07:42,465 INFO (stateListener|82) [BDBEnvironment.getReplicationGroupAdmin():231] addresses is empty
RuntimeLogger 2024-09-09 09:07:42,466 ERROR (stateListener|82) [Env.checkCurrentNodeExist():1932] current node doriscluster-sample-storageclass1-fe-1.doriscluster-sample-storageclass1-fe-internal.doris.svc.cluster.local:9010 is not added to the cluster, will exit. Your FE IP maybe changed, please set 'priority_networks' config in fe.conf properly.

DorisCluster的yaml如下:

apiVersion: doris.selectdb.com/v1
kind: DorisCluster
metadata:
  labels:
    app.kubernetes.io/name: doriscluster
    app.kubernetes.io/instance: doriscluster-sample-storageclass
    app.kubernetes.io/part-of: doris-operator
  name: doriscluster-sample-storageclass1
spec:
  feSpec:
    configMapInfo:
      configMapName: fe-conf
      resolveKey: fe.conf
    replicas: 3
    service:
      type: LoadBalancer
    image: registry.XXXX.XX/doris/doris.fe:v3.0.1
    systemInitialization:
      initImage: registry.XXXX.XX/doris/doris.alpine:20240905
    imagePullSecrets:
     - name: admin-secret
    limits:
      cpu: 8
      memory: 16Gi
    requests:
      cpu: 8
      memory: 16Gi
    persistentVolumes:
    - name: fe-meta
      mountPath: /opt/apache-doris/fe/doris-meta
      persistentVolumeClaimSpec:
        accessModes:
        - ReadWriteOnce
        resources:
            storage: 50Gi
    - name: fe-log
      mountPath: /opt/apache-doris/fe/log
      persistentVolumeClaimSpec:
        # when use specific storageclass, the storageClassName should reConfig, example as annotation.
        storageClassName: course-nfs-storage
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 50Gi
  beSpec:
    configMapInfo:
      configMapName: be-conf
      resolveKey: be.conf
    replicas: 3
    service:
      type: ClusterIP
    image: registry.XXXX.XX/doris/doris.be:v3.0.1
    systemInitialization:
      initImage: registry.XXXX.XX/doris/doris.alpine:20240905
    imagePullSecrets:
     - name: admin-secret
    limits:
      cpu: 8
      memory: 16Gi
    requests:
      cpu: 8
      memory: 16Gi
    persistentVolumes:
    - name: be-storage
      mountPath: /opt/apache-doris/be/storage
      persistentVolumeClaimSpec:
        # when use specific storageclass, the storageClassName should reConfig, example as annotation.
        storageClassName: course-nfs-storage
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 500Gi
    - name: belog
      mountPath: /opt/apache-doris/be/log
      persistentVolumeClaimSpec:
        # when use specific storageclass, the storageClassName should reConfig, example as annotation.
        storageClassName: course-nfs-storage
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 50Gi
---

apiVersion: v1
kind: ConfigMap
metadata:
  name: fe-conf
  labels:
    app.kubernetes.io/component: fe
data:
  fe.conf: |
    #BUILD_TYPE = k8s
    #priority_networks = 10.244.0.0/16
    CUR_DATE=`date +%Y%m%d-%H%M%S`
    # the output dir of stderr and stdout
    LOG_DIR = ${DORIS_HOME}/log
    JAVA_OPTS="-Djavax.security.auth.useSubjectCredsOnly=false -Xss4m -Xmx8192m -XX:+UseMembar -XX:SurvivorRatio=8 -XX:MaxTenuringThreshold=7 -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSClassUnloadingEnabled -XX:-CMSParallelRemarkEnabled -XX:CMSInitiatingOccupancyFraction=80 -XX:SoftRefLRUPolicyMSPerMB=0 -Xloggc:$DORIS_HOME/log/fe.gc.log.$CUR_DATE"
    # For jdk 9+, this JAVA_OPTS will be used as default JVM options
    JAVA_OPTS_FOR_JDK_17="-Djavax.security.auth.useSubjectCredsOnly=false -Xmx8192m -Xms8192m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=$LOG_DIR -Xlog:gc*:$LOG_DIR/fe.gc.log.$CUR_DATE:time,uptime:filecount=10,filesize=50M --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens java.base/jdk.internal.ref=ALL-UNNAMED"
    # INFO, WARN, ERROR, FATAL
    sys_log_level = INFO
    # NORMAL, BRIEF, ASYNC
    sys_log_mode = NORMAL
    # Default dirs to put jdbc drivers,default value is ${DORIS_HOME}/jdbc_drivers
    # jdbc_drivers_dir = ${DORIS_HOME}/jdbc_drivers
    # meta_dir = /opt/apache-doris/fe/doris-meta
    http_port = 8030
    rpc_port = 9020
    query_port = 9030
    edit_log_port = 9010
    enable_fqdn_mode = true
    arrow_flight_sql_port = -1

---

apiVersion: v1
kind: ConfigMap
metadata:
  name: be-conf
  labels:
    app.kubernetes.io/component: be
data:
  be.conf: |
    #BUILD_TYPE = k8s
    #priority_networks = 10.244.0.0/16
    CUR_DATE=`date +%Y%m%d-%H%M%S`
    PPROF_TMPDIR="$DORIS_HOME/log/"
    JAVA_OPTS="-Xmx1024m -DlogPath=$LOG_DIR/jni.log -Xloggc:$LOG_DIR/be.gc.log.$CUR_DATE -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=50M -Djavax.security.auth.useSubjectCredsOnly=false -Dsun.security.krb5.debug=true -Dsun.java.command=DorisBE -XX:-CriticalJNINatives"
    # For jdk 9+, this JAVA_OPTS will be used as default JVM options
    JAVA_OPTS_FOR_JDK_17="-Xmx1024m -DlogPath=$LOG_DIR/jni.log -Xlog:gc*:$LOG_DIR/be.gc.log.$CUR_DATE:time,uptime:filecount=10,filesize=50M -Djavax.security.auth.useSubjectCredsOnly=false -Dsun.security.krb5.debug=true -Dsun.java.command=DorisBE -XX:-CriticalJNINatives -XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-opens=java.management/sun.management=ALL-UNNAMED"
    # since 1.2, the JAVA_HOME need to be set to run BE process.
    # JAVA_HOME=/path/to/jdk/
    # https://github.com/apache/doris/blob/master/docs/zh-CN/community/developer-guide/debug-tool.md#jemalloc-heap-profile
    # https://jemalloc.net/jemalloc.3.html
    JEMALLOC_CONF="percpu_arena:percpu,background_thread:true,metadata_thp:auto,muzzy_decay_ms:15000,dirty_decay_ms:15000,oversize_threshold:0,prof:false,lg_prof_interval:32,lg_prof_sample:19,prof_gdump:false,prof_accum:false,prof_leak:false,prof_final:false"
    JEMALLOC_PROF_PRFIX=""
    # INFO, WARNING, ERROR, FATAL
    sys_log_level = INFO
    # ports for admin, web, heartbeat service
    be_port = 9060
    webserver_port = 8040
    heartbeat_service_port = 9050
    brpc_port = 8060
    enable_https = false
    # Default to turn off aws sdk log, because aws sdk errors that need to be cared will be output through Doris logs
    aws_log_level=0
    # ## If you are not running in aws cloud, you can disable EC2 metadata
    AWS_EC2_METADATA_DISABLED=true

2 Answers

你看下你这个启动命令是怎样的?priority_networks 设置了吗
image.png