【已记录】doris 从2.0.1升级到2.1.0,部分 jdk 版本会导致崩溃问题

Viewed 147

在doris从版本2.0.1升级到2.1.0过程中,采用环境openjdk11。
启动be过程中,be崩溃,产生core文件,日志:
version doris-2.1.0-rc11(SSE4.2) RELEASE (build git://VM-0-32-ubuntu@91efb6a43d8a6c92cdd9d1729806a6bb8134fc84)
Built on Mon, 04 Mar 2024 23:09:21 CST by VM-0-32-ubuntu
启动fe过程。fe无法启动,日志:
java.io.IOException: java.lang.reflect.InvocationTargetException
at org.apache.doris.persist.meta.MetaReader.read(MetaReader.java:129) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.Env.loadImage(Env.java:1882) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.Env.initialize(Env.java:1002) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.DorisFE.start(DorisFE.java:168) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.DorisFE.main(DorisFE.java:83) ~[doris-fe.jar:1.2-SNAPSHOT]
Caused by: java.lang.reflect.InvocationTargetException
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
at org.apache.doris.persist.meta.MetaReader.read(MetaReader.java:126) ~[doris-fe.jar:1.2-SNAPSHOT]
... 4 more
Caused by: com.google.gson.JsonIOException: Failed making field 'jdk.internal.module.ServicesCatalog#map' accessible; either increase its visibility or write a custom TypeAdapter for its declaring type.
at com.google.gson.internal.reflect.ReflectionHelper.makeAccessible(ReflectionHelper.java:38) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.getBoundFields(ReflectiveTypeAdapterFactory.java:286) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.create(ReflectiveTypeAdapterFactory.java:130) ~[gson-2.10.1.jar:?]
at com.google.gson.Gson.getDelegateAdapter(Gson.java:652) ~[gson-2.10.1.jar:?]
at org.apache.doris.persist.gson.GsonUtils$PostProcessTypeAdapterFactory.create(GsonUtils.java:575) ~[doris-fe.jar:1.2-SNAPSHOT]
at com.google.gson.Gson.getAdapter(Gson.java:556) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.createBoundField(ReflectiveTypeAdapterFactory.java:160) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.getBoundFields(ReflectiveTypeAdapterFactory.java:294) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.create(ReflectiveTypeAdapterFactory.java:130) ~[gson-2.10.1.jar:?]
at com.google.gson.Gson.getDelegateAdapter(Gson.java:652) ~[gson-2.10.1.jar:?]
at org.apache.doris.persist.gson.GsonUtils$PostProcessTypeAdapterFactory.create(GsonUtils.java:575) ~[doris-fe.jar:1.2-SNAPSHOT]
at com.google.gson.Gson.getAdapter(Gson.java:556) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.createBoundField(ReflectiveTypeAdapterFactory.java:160) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.getBoundFields(ReflectiveTypeAdapterFactory.java:294) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.create(ReflectiveTypeAdapterFactory.java:130) ~[gson-2.10.1.jar:?]
at com.google.gson.Gson.getDelegateAdapter(Gson.java:652) ~[gson-2.10.1.jar:?]
at org.apache.doris.persist.gson.GsonUtils$PostProcessTypeAdapterFactory.create(GsonUtils.java:575) ~[doris-fe.jar:1.2-SNAPSHOT]
at com.google.gson.Gson.getAdapter(Gson.java:556) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.createBoundField(ReflectiveTypeAdapterFactory.java:160) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.getBoundFields(ReflectiveTypeAdapterFactory.java:294) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.create(ReflectiveTypeAdapterFactory.java:130) ~[gson-2.10.1.jar:?]
at com.google.gson.Gson.getDelegateAdapter(Gson.java:652) ~[gson-2.10.1.jar:?]
at org.apache.doris.persist.gson.GsonUtils$PostProcessTypeAdapterFactory.create(GsonUtils.java:575) ~[doris-fe.jar:1.2-SNAPSHOT]
at com.google.gson.Gson.getAdapter(Gson.java:556) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.createBoundField(ReflectiveTypeAdapterFactory.java:160) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.getBoundFields(ReflectiveTypeAdapterFactory.java:294) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.create(ReflectiveTypeAdapterFactory.java:130) ~[gson-2.10.1.jar:?]
at com.google.gson.Gson.getDelegateAdapter(Gson.java:652) ~[gson-2.10.1.jar:?]
at org.apache.doris.persist.gson.GsonUtils$PostProcessTypeAdapterFactory.create(GsonUtils.java:575) ~[doris-fe.jar:1.2-SNAPSHOT]
at com.google.gson.Gson.getAdapter(Gson.java:556) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.createBoundField(ReflectiveTypeAdapterFactory.java:160) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.getBoundFields(ReflectiveTypeAdapterFactory.java:294) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.create(ReflectiveTypeAdapterFactory.java:130) ~[gson-2.10.1.jar:?]
at com.google.gson.Gson.getDelegateAdapter(Gson.java:652) ~[gson-2.10.1.jar:?]
at org.apache.doris.persist.gson.GsonUtils$PostProcessTypeAdapterFactory.create(GsonUtils.java:575) ~[doris-fe.jar:1.2-SNAPSHOT]
at com.google.gson.Gson.getAdapter(Gson.java:556) ~[gson-2.10.1.jar:?]
at com.google.gson.Gson.fromJson(Gson.java:1226) ~[gson-2.10.1.jar:?]
at com.google.gson.Gson.fromJson(Gson.java:1137) ~[gson-2.10.1.jar:?]
at com.google.gson.Gson.fromJson(Gson.java:1047) ~[gson-2.10.1.jar:?]
at com.google.gson.Gson.fromJson(Gson.java:982) ~[gson-2.10.1.jar:?]
at org.apache.doris.resource.workloadgroup.WorkloadGroupMgr.read(WorkloadGroupMgr.java:150) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.Env.loadWorkloadGroups(Env.java:2129) ~[doris-fe.jar:1.2-SNAPSHOT]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
at org.apache.doris.persist.meta.MetaReader.read(MetaReader.java:126) ~[doris-fe.jar:1.2-SNAPSHOT]
... 4 more
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field private final java.util.Map jdk.internal.module.ServicesCatalog.map accessible: module java.base does not "opens jdk.internal.module" to unnamed module @530ee28b
at java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:340) ~[?:?]
at java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:280) ~[?:?]
at java.lang.reflect.Field.checkCanSetAccessible(Field.java:176) ~[?:?]
at java.lang.reflect.Field.setAccessible(Field.java:170) ~[?:?]
at com.google.gson.internal.reflect.ReflectionHelper.makeAccessible(ReflectionHelper.java:35) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.getBoundFields(ReflectiveTypeAdapterFactory.java:286) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.create(ReflectiveTypeAdapterFactory.java:130) ~[gson-2.10.1.jar:?]
at com.google.gson.Gson.getDelegateAdapter(Gson.java:652) ~[gson-2.10.1.jar:?]
at org.apache.doris.persist.gson.GsonUtils$PostProcessTypeAdapterFactory.create(GsonUtils.java:575) ~[doris-fe.jar:1.2-SNAPSHOT]
at com.google.gson.Gson.getAdapter(Gson.java:556) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.createBoundField(ReflectiveTypeAdapterFactory.java:160) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.getBoundFields(ReflectiveTypeAdapterFactory.java:294) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.create(ReflectiveTypeAdapterFactory.java:130) ~[gson-2.10.1.jar:?]
at com.google.gson.Gson.getDelegateAdapter(Gson.java:652) ~[gson-2.10.1.jar:?]
at org.apache.doris.persist.gson.GsonUtils$PostProcessTypeAdapterFactory.create(GsonUtils.java:575) ~[doris-fe.jar:1.2-SNAPSHOT]
at com.google.gson.Gson.getAdapter(Gson.java:556) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.createBoundField(ReflectiveTypeAdapterFactory.java:160) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.getBoundFields(ReflectiveTypeAdapterFactory.java:294) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.create(ReflectiveTypeAdapterFactory.java:130) ~[gson-2.10.1.jar:?]
at com.google.gson.Gson.getDelegateAdapter(Gson.java:652) ~[gson-2.10.1.jar:?]
at org.apache.doris.persist.gson.GsonUtils$PostProcessTypeAdapterFactory.create(GsonUtils.java:575) ~[doris-fe.jar:1.2-SNAPSHOT]
at com.google.gson.Gson.getAdapter(Gson.java:556) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.createBoundField(ReflectiveTypeAdapterFactory.java:160) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.getBoundFields(ReflectiveTypeAdapterFactory.java:294) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.create(ReflectiveTypeAdapterFactory.java:130) ~[gson-2.10.1.jar:?]
at com.google.gson.Gson.getDelegateAdapter(Gson.java:652) ~[gson-2.10.1.jar:?]
at org.apache.doris.persist.gson.GsonUtils$PostProcessTypeAdapterFactory.create(GsonUtils.java:575) ~[doris-fe.jar:1.2-SNAPSHOT]
at com.google.gson.Gson.getAdapter(Gson.java:556) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.createBoundField(ReflectiveTypeAdapterFactory.java:160) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.getBoundFields(ReflectiveTypeAdapterFactory.java:294) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.create(ReflectiveTypeAdapterFactory.java:130) ~[gson-2.10.1.jar:?]
at com.google.gson.Gson.getDelegateAdapter(Gson.java:652) ~[gson-2.10.1.jar:?]
at org.apache.doris.persist.gson.GsonUtils$PostProcessTypeAdapterFactory.create(GsonUtils.java:575) ~[doris-fe.jar:1.2-SNAPSHOT]
at com.google.gson.Gson.getAdapter(Gson.java:556) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.createBoundField(ReflectiveTypeAdapterFactory.java:160) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.getBoundFields(ReflectiveTypeAdapterFactory.java:294) ~[gson-2.10.1.jar:?]
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory.create(ReflectiveTypeAdapterFactory.java:130) ~[gson-2.10.1.jar:?]
at com.google.gson.Gson.getDelegateAdapter(Gson.java:652) ~[gson-2.10.1.jar:?]
at org.apache.doris.persist.gson.GsonUtils$PostProcessTypeAdapterFactory.create(GsonUtils.java:575) ~[doris-fe.jar:1.2-SNAPSHOT]
at com.google.gson.Gson.getAdapter(Gson.java:556) ~[gson-2.10.1.jar:?]
at com.google.gson.Gson.fromJson(Gson.java:1226) ~[gson-2.10.1.jar:?]
at com.google.gson.Gson.fromJson(Gson.java:1137) ~[gson-2.10.1.jar:?]
at com.google.gson.Gson.fromJson(Gson.java:1047) ~[gson-2.10.1.jar:?]
at com.google.gson.Gson.fromJson(Gson.java:982) ~[gson-2.10.1.jar:?]
at org.apache.doris.resource.workloadgroup.WorkloadGroupMgr.read(WorkloadGroupMgr.java:150) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.Env.loadWorkloadGroups(Env.java:2129) ~[doris-fe.jar:1.2-SNAPSHOT]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
at org.apache.doris.persist.meta.MetaReader.read(MetaReader.java:126) ~[doris-fe.jar:1.2-SNAPSHOT]
... 4 more

切换成jdk1.8,正常启动。

3 Answers

doris在2.1,还是支持jdk 8.

新的jdk由于支撑模块化等一些新功能,在反射的api支持上做了一些和jdk 8 不兼容的改动,所以,在使用doris 2.x的时候,请务必使用JDK 8.

预计在下一个大版本,我们会将jdk升级到jdk 17,以便利用最新的jdk技术。jdk 17也是oracle可以免费商用的一个LTS版本。

请问 be core 时的 be.out 有吗?升级的时候在跑什么负载

当年手贱用jdk17部署的doris,各种报错,老老实实换回了JDK8,世界清净了。