2024-04-10 15:59:31.616 [SimpleAsyncTaskExecutor-917] INFO c.s.e.m.service.component.agentjob.JobExecutor - execute ClusterUpgrade job 2 uuid abeca9c4a402433a877fdf9c9fa05cd0.
2024-04-10 15:59:31.620 [SimpleAsyncTaskExecutor-917] INFO c.s.e.m.service.component.agentjob.JobExecutor - before operate for ClusterUpgrade job 2 uuid abeca9c4a402433a877fdf9c9fa05cd0.
2024-04-10 15:59:31.620 [SimpleAsyncTaskExecutor-917] INFO c.s.e.m.service.component.agentjob.JobExecutor - get ClusterUpgrade job 2 uuid abeca9c4a402433a877fdf9c9fa05cd0.
2024-04-10 15:59:31.623 [SimpleAsyncTaskExecutor-917] INFO c.s.e.m.service.component.agentjob.JobExecutor - get ClusterUpgrade job 2 uuid abeca9c4a402433a877fdf9c9fa05cd0 stage DownloadPackage.
2024-04-10 15:59:31.623 [SimpleAsyncTaskExecutor-917] INFO c.s.e.m.service.component.agentjob.JobExecutor - get ClusterUpgrade job 2 uuid abeca9c4a402433a877fdf9c9fa05cd0 stage DownloadPackage.
2024-04-10 15:59:31.624 [pool-4-thread-27] INFO c.s.e.m.service.component.agentjob.stage.JobStage - [Start executing stage 3 - DownloadPackage]
2024-04-10 15:59:31.624 [SimpleAsyncTaskExecutor-917] INFO c.s.e.m.service.component.agentjob.JobExecutor - get ClusterUpgrade job 2 uuid abeca9c4a402433a877fdf9c9fa05cd0 stage DownloadPackage.
2024-04-10 15:59:31.624 [pool-4-thread-28] INFO c.s.e.m.service.component.agentjob.stage.JobStage - [Start executing stage 3 - DownloadPackage]
2024-04-10 15:59:31.625 [pool-4-thread-28] INFO c.s.e.m.service.component.agentjob.stage.JobStage - start job order 3 DownloadPackage stage 10
2024-04-10 15:59:31.625 [pool-4-thread-27] INFO c.s.e.m.service.component.agentjob.stage.JobStage - start job order 3 DownloadPackage stage 9
2024-04-10 15:59:31.625 [pool-4-thread-28] INFO c.s.e.m.service.component.agentjob.stage.JobStage - update stage 10 status running start.
2024-04-10 15:59:31.625 [pool-4-thread-27] INFO c.s.e.m.service.component.agentjob.stage.JobStage - update stage 9 status running start.
2024-04-10 15:59:31.625 [pool-4-thread-29] INFO c.s.e.m.service.component.agentjob.stage.JobStage - [Start executing stage 3 - DownloadPackage]
2024-04-10 15:59:31.631 [pool-4-thread-29] INFO c.s.e.m.service.component.agentjob.stage.JobStage - start job order 3 DownloadPackage stage 11
2024-04-10 15:59:31.631 [pool-4-thread-29] INFO c.s.e.m.service.component.agentjob.stage.JobStage - update stage 11 status running start.
2024-04-10 15:59:31.633 [pool-4-thread-28] INFO c.s.e.m.s.c.agentjob.cache.JobStorageManager - Update stage 10 cache
2024-04-10 15:59:31.633 [pool-4-thread-28] INFO c.s.e.m.service.component.agentjob.stage.JobStage - update stage 10 status running end.
2024-04-10 15:59:31.633 [pool-4-thread-28] INFO c.s.e.m.service.component.agentjob.stage.JobStage - download stage, calculate package md5 start.
2024-04-10 15:59:31.633 [pool-4-thread-27] INFO c.s.e.m.s.c.agentjob.cache.JobStorageManager - Update stage 9 cache
2024-04-10 15:59:31.633 [pool-4-thread-27] INFO c.s.e.m.service.component.agentjob.stage.JobStage - update stage 9 status running end.
2024-04-10 15:59:31.633 [pool-4-thread-27] INFO c.s.e.m.service.component.agentjob.stage.JobStage - download stage, calculate package md5 start.
2024-04-10 15:59:31.634 [pool-4-thread-29] INFO c.s.e.m.s.c.agentjob.cache.JobStorageManager - Update stage 11 cache
2024-04-10 15:59:31.634 [pool-4-thread-29] INFO c.s.e.m.service.component.agentjob.stage.JobStage - update stage 11 status running end.
2024-04-10 15:59:31.634 [pool-4-thread-29] INFO c.s.e.m.service.component.agentjob.stage.JobStage - download stage, calculate package md5 start.
2024-04-10 15:59:36.616 [SimpleAsyncTaskExecutor-918] INFO c.s.e.m.service.component.agentjob.JobExecutor - execute ClusterUpgrade job 2 uuid abeca9c4a402433a877fdf9c9fa05cd0.
2024-04-10 15:59:36.616 [SimpleAsyncTaskExecutor-918] INFO c.s.e.m.service.component.agentjob.JobExecutor - get ClusterUpgrade job 2 uuid abeca9c4a402433a877fdf9c9fa05cd0 stage DownloadPackage.
2024-04-10 15:59:36.616 [SimpleAsyncTaskExecutor-918] INFO c.s.e.m.service.component.agentjob.JobExecutor - get ClusterUpgrade job 2 uuid abeca9c4a402433a877fdf9c9fa05cd0 stage DownloadPackage.
2024-04-10 15:59:36.616 [SimpleAsyncTaskExecutor-918] INFO c.s.e.m.service.component.agentjob.JobExecutor - get ClusterUpgrade job 2 uuid abeca9c4a402433a877fdf9c9fa05cd0 stage DownloadPackage.
2024-04-10 15:59:41.616 [SimpleAsyncTaskExecutor-920] INFO c.s.e.m.service.component.agentjob.JobExecutor - execute ClusterUpgrade job 2 uuid abeca9c4a402433a877fdf9c9fa05cd0.
2024-04-10 15:59:41.616 [SimpleAsyncTaskExecutor-920] INFO c.s.e.m.service.component.agentjob.JobExecutor - get ClusterUpgrade job 2 uuid abeca9c4a402433a877fdf9c9fa05cd0 stage DownloadPackage.
2024-04-10 15:59:41.616 [SimpleAsyncTaskExecutor-920] INFO c.s.e.m.service.component.agentjob.JobExecutor - get ClusterUpgrade job 2 uuid abeca9c4a402433a877fdf9c9fa05cd0 stage DownloadPackage.
2024-04-10 15:59:41.616 [SimpleAsyncTaskExecutor-920] INFO c.s.e.m.service.component.agentjob.JobExecutor - get ClusterUpgrade job 2 uuid abeca9c4a402433a877fdf9c9fa05cd0 stage DownloadPackage.
2024-04-10 15:59:43.144 [pool-4-thread-28] INFO c.s.e.m.service.component.agentjob.stage.JobStage - package /data/software/apache-doris-2.0.8-bin-x64.tar.gz md5 1ac84c9b509b48ebaf6dec38fe9faa8d .
2024-04-10 15:59:43.144 [pool-4-thread-29] INFO c.s.e.m.service.component.agentjob.stage.JobStage - package /data/software/apache-doris-2.0.8-bin-x64.tar.gz md5 1ac84c9b509b48ebaf6dec38fe9faa8d .
2024-04-10 15:59:43.144 [pool-4-thread-27] INFO c.s.e.m.service.component.agentjob.stage.JobStage - package /data/software/apache-doris-2.0.8-bin-x64.tar.gz md5 1ac84c9b509b48ebaf6dec38fe9faa8d .
2024-04-10 15:59:43.145 [pool-4-thread-29] INFO c.s.e.m.s.c.agentjob.cache.JobStorageManager - Update stage 11 cache
2024-04-10 15:59:43.145 [pool-4-thread-27] INFO c.s.e.m.s.c.agentjob.cache.JobStorageManager - Update stage 9 cache
2024-04-10 15:59:43.145 [pool-4-thread-29] INFO c.s.e.m.service.component.agentjob.stage.JobStage - download stage, calculate package md5 end.
2024-04-10 15:59:43.145 [pool-4-thread-28] INFO c.s.e.m.s.c.agentjob.cache.JobStorageManager - Update stage 10 cache
2024-04-10 15:59:43.145 [pool-4-thread-27] INFO c.s.e.m.service.component.agentjob.stage.JobStage - download stage, calculate package md5 end.
2024-04-10 15:59:43.145 [pool-4-thread-28] INFO c.s.e.m.service.component.agentjob.stage.JobStage - download stage, calculate package md5 end.
2024-04-10 15:59:43.145 [pool-4-thread-28] INFO c.s.e.m.s.component.agent.AgentWebSocketClient - connect to agent id 2 ip 192.168.10.180 command websocket, uri ws://192.168.10.180:8972/command
2024-04-10 15:59:43.145 [pool-4-thread-29] INFO c.s.e.m.s.component.agent.AgentWebSocketClient - connect to agent id 3 ip 192.168.10.162 command websocket, uri ws://192.168.10.162:8972/command
2024-04-10 15:59:43.145 [pool-4-thread-27] INFO c.s.e.m.s.component.agent.AgentWebSocketClient - connect to agent id 1 ip 192.168.10.189 command websocket, uri ws://192.168.10.189:8972/command
2024-04-10 15:59:43.154 [pool-4-thread-29] INFO c.s.e.m.s.component.agent.AgentWebSocketClient - send agent id 3 ip 192.168.10.162 request {"agent_id":"agent-gwu5notcbprgstaxhx","job_id":2,"job_uuid":"abeca9c4a402433a877fdf9c9fa05cd0","manager_id":"manager-siqpd4tb9r62pxs5kv","node":{"be_port":9060,"be_role":"mix","brpc_port":8060,"cluster_id":1,"deploy_dir":"/data/doris/be","edit_log_port":0,"heartbeat_port":9050,"http_port":0,"id":3,"ip":"192.168.10.162","jdbc_port":0,"keep_alive":true,"log_dir":"/data/doris/be/log","module_name":"Be","priority_networks":"192.168.10.0/24","resource_node_id":3,"rpc_port":0,"status":"Upgrading","storage_dir":"/data/doris/be/storage","webserver_port":8040,"webui_memory":0,"webui_port":0},"resource":{"download_dir":"/data/doris/upgrade","ips":["fe80:0:0:0:988b:3bff:fe6e:93a5%veth0e51631","fe80:0:0:0:3cec:a5ff:fe40:78ab%veth4de39cb7","fe80:0:0:0:ec6a:22ff:fe04:af2f%vethf885f6b3","fe80:0:0:0:acfd:52ff:fe7c:354c%veth2c42ec3b","fe80:0:0:0:3830:6fff:fe62:148a%cni0","10.244.0.1","fe80:0:0:0:90c5:87ff:febd:98fd%flannel.1","10.244.0.0","fe80:0:0:0:42:afff:fea0:fc99%docker0","172.17.0.1","fe80:0:0:0:250:56ff:fe95:b267%eth0","192.168.10.189"],"modules":["Be"],"package_md5_sum":"1ac84c9b509b48ebaf6dec38fe9faa8d","package_name":"apache-doris-2.0.8-bin-x64.tar.gz","package_path":"/data/software","port":8004},"stage_id":11,"stage_order":3,"start_time":1712732529119,"timeout":1800,"type":"DownloadPackage"}.
2024-04-10 15:59:43.154 [pool-4-thread-28] INFO c.s.e.m.s.component.agent.AgentWebSocketClient - send agent id 2 ip 192.168.10.180 request {"agent_id":"agent-xl4f34tcvbonuj9dj7","job_id":2,"job_uuid":"abeca9c4a402433a877fdf9c9fa05cd0","manager_id":"manager-siqpd4tb9r62pxs5kv","node":{"be_port":9060,"be_role":"mix","brpc_port":8060,"cluster_id":1,"deploy_dir":"/data/doris/be","edit_log_port":0,"heartbeat_port":9050,"http_port":0,"id":2,"ip":"192.168.10.180","jdbc_port":0,"keep_alive":true,"log_dir":"/data/doris/be/log","module_name":"Be","priority_networks":"192.168.10.0/24","resource_node_id":2,"rpc_port":0,"status":"Upgrading","storage_dir":"/data/doris/be/storage","webserver_port":8040,"webui_memory":0,"webui_port":0},"resource":{"download_dir":"/data/doris/upgrade","ips":["fe80:0:0:0:988b:3bff:fe6e:93a5%veth0e51631","fe80:0:0:0:3cec:a5ff:fe40:78ab%veth4de39cb7","fe80:0:0:0:ec6a:22ff:fe04:af2f%vethf885f6b3","fe80:0:0:0:acfd:52ff:fe7c:354c%veth2c42ec3b","fe80:0:0:0:3830:6fff:fe62:148a%cni0","10.244.0.1","fe80:0:0:0:90c5:87ff:febd:98fd%flannel.1","10.244.0.0","fe80:0:0:0:42:afff:fea0:fc99%docker0","172.17.0.1","fe80:0:0:0:250:56ff:fe95:b267%eth0","192.168.10.189"],"modules":["Be"],"package_md5_sum":"1ac84c9b509b48ebaf6dec38fe9faa8d","package_name":"apache-doris-2.0.8-bin-x64.tar.gz","package_path":"/data/software","port":8004},"stage_id":10,"stage_order":3,"start_time":1712732529117,"timeout":1800,"type":"DownloadPackage"}.
2024-04-10 15:59:43.155 [pool-4-thread-27] INFO c.s.e.m.s.component.agent.AgentWebSocketClient - send agent id 1 ip 192.168.10.189 request {"agent_id":"agent-bw253ejm0376rb3w2l","job_id":2,"job_uuid":"abeca9c4a402433a877fdf9c9fa05cd0","manager_id":"manager-siqpd4tb9r62pxs5kv","node":{"be_port":0,"brpc_port":0,"cluster_id":1,"deploy_dir":"/data/doris/fe","edit_log_port":9010,"heartbeat_port":0,"http_port":8030,"id":1,"ip":"192.168.10.189","is_master":true,"jdbc_port":9030,"keep_alive":true,"log_dir":"/data/doris/fe/log","meta_dir":"/data/doris/fe/meta","module_name":"Fe","priority_networks":"192.168.10.0/24","resource_node_id":1,"role":"Follower","rpc_port":9020,"status":"Upgrading","webserver_port":0,"webui_memory":0,"webui_port":0},"resource":{"download_dir":"/data/doris/upgrade","ips":["fe80:0:0:0:988b:3bff:fe6e:93a5%veth0e51631","fe80:0:0:0:3cec:a5ff:fe40:78ab%veth4de39cb7","fe80:0:0:0:ec6a:22ff:fe04:af2f%vethf885f6b3","fe80:0:0:0:acfd:52ff:fe7c:354c%veth2c42ec3b","fe80:0:0:0:3830:6fff:fe62:148a%cni0","10.244.0.1","fe80:0:0:0:90c5:87ff:febd:98fd%flannel.1","10.244.0.0","fe80:0:0:0:42:afff:fea0:fc99%docker0","172.17.0.1","fe80:0:0:0:250:56ff:fe95:b267%eth0","192.168.10.189"],"modules":["Fe","Be"],"package_md5_sum":"1ac84c9b509b48ebaf6dec38fe9faa8d","package_name":"apache-doris-2.0.8-bin-x64.tar.gz","package_path":"/data/software","port":8004},"stage_id":9,"stage_order":3,"start_time":1712732529110,"timeout":1800,"type":"DownloadPackage"}.
2024-04-10 15:59:43.159 [WebSocketClient-AsyncIO-8] INFO c.s.e.m.s.c.a.m.AgentTaskExecuteWebSocketHandler - handle message of job abeca9c4a402433a877fdf9c9fa05cd0 from agent id 1 ip 192.168.10.189.
2024-04-10 15:59:43.159 [WebSocketClient-AsyncIO-8] INFO c.s.e.m.s.c.a.m.AgentTaskExecuteWebSocketHandler - received agent id 1 ip 192.168.10.189 command message {"code":1,"message":"failed to download [apache-doris-2.0.8-bin-x64.tar.gz] into path [/data/doris/upgrade/apache-doris-2.0.8-bin-x64.tar.gz], failed to new download request, parse \"http://fe80:0:0:0:988b:3bff:fe6e:93a5%veth0e51631:8004/api/download/doris-package?fileName=apache-doris-2.0.8-bin-x64.tar.gz\": invalid URL escape \"%ve\""}.
2024-04-10 15:59:43.159 [pool-4-thread-27] INFO c.s.e.m.s.c.a.m.AgentTaskExecuteWebSocketHandler - agent id 1 ip 192.168.10.189 reached timeout.
2024-04-10 15:59:43.160 [pool-4-thread-27] INFO c.s.e.m.service.component.agentjob.stage.JobStage - update stage 9 status complete Fail start.
2024-04-10 15:59:43.161 [pool-4-thread-27] INFO c.s.e.m.s.c.agentjob.cache.JobStorageManager - Update stage 9 cache
2024-04-10 15:59:43.161 [pool-4-thread-27] INFO c.s.e.m.service.component.agentjob.stage.JobStage - update stage 9 status complete Fail end.
2024-04-10 15:59:43.161 [pool-4-thread-27] INFO c.s.e.m.service.component.agentjob.stage.JobStage - the stage 9 not need to update cluster node status.
2024-04-10 15:59:44.660 [WebSocketClient-AsyncIO-3] INFO c.s.e.m.s.c.a.m.AgentTaskExecuteWebSocketHandler - handle message of job abeca9c4a402433a877fdf9c9fa05cd0 from agent id 2 ip 192.168.10.180.
2024-04-10 15:59:44.660 [WebSocketClient-AsyncIO-3] INFO c.s.e.m.s.c.a.m.AgentTaskExecuteWebSocketHandler - received agent id 2 ip 192.168.10.180 command message {"code":1,"message":"failed to download [apache-doris-2.0.8-bin-x64.tar.gz] into path [/data/doris/upgrade/apache-doris-2.0.8-bin-x64.tar.gz], failed to new download request, parse \"http://fe80:0:0:0:250:56ff:fe95:b267%eth0:8004/api/download/doris-package?fileName=apache-doris-2.0.8-bin-x64.tar.gz\": invalid URL escape \"%et\""}.
2024-04-10 15:59:44.660 [pool-4-thread-28] INFO c.s.e.m.s.c.a.m.AgentTaskExecuteWebSocketHandler - agent id 2 ip 192.168.10.180 reached timeout.
2024-04-10 15:59:44.660 [pool-4-thread-28] INFO c.s.e.m.service.component.agentjob.stage.JobStage - update stage 10 status complete Fail start.
2024-04-10 15:59:44.661 [pool-4-thread-28] INFO c.s.e.m.s.c.agentjob.cache.JobStorageManager - Update stage 10 cache
2024-04-10 15:59:44.661 [pool-4-thread-28] INFO c.s.e.m.service.component.agentjob.stage.JobStage - update stage 10 status complete Fail end.
2024-04-10 15:59:44.661 [pool-4-thread-28] INFO c.s.e.m.service.component.agentjob.stage.JobStage - the stage 10 not need to update cluster node status.
2024-04-10 15:59:44.662 [WebSocketClient-AsyncIO-2] INFO c.s.e.m.s.c.a.m.AgentTaskExecuteWebSocketHandler - handle message of job abeca9c4a402433a877fdf9c9fa05cd0 from agent id 3 ip 192.168.10.162.
2024-04-10 15:59:44.662 [WebSocketClient-AsyncIO-2] INFO c.s.e.m.s.c.a.m.AgentTaskExecuteWebSocketHandler - received agent id 3 ip 192.168.10.162 command message {"code":1,"message":"failed to download [apache-doris-2.0.8-bin-x64.tar.gz] into path [/data/doris/upgrade/apache-doris-2.0.8-bin-x64.tar.gz], failed to new download request, parse \"http://fe80:0:0:0:250:56ff:fe95:b267%eth0:8004/api/download/doris-package?fileName=apache-doris-2.0.8-bin-x64.tar.gz\": invalid URL escape \"%et\""}.
2024-04-10 15:59:44.662 [pool-4-thread-29] INFO c.s.e.m.s.c.a.m.AgentTaskExecuteWebSocketHandler - agent id 3 ip 192.168.10.162 reached timeout.
2024-04-10 15:59:44.662 [pool-4-thread-29] INFO c.s.e.m.service.component.agentjob.stage.JobStage - update stage 11 status complete Fail start.
2024-04-10 15:59:44.663 [pool-4-thread-29] INFO c.s.e.m.s.c.agentjob.cache.JobStorageManager - Update stage 11 cache
2024-04-10 15:59:44.663 [pool-4-thread-29] INFO c.s.e.m.service.component.agentjob.stage.JobStage - update stage 11 status complete Fail end.
2024-04-10 15:59:44.663 [pool-4-thread-29] INFO c.s.e.m.service.component.agentjob.stage.JobStage - the stage 11 not need to update cluster node status.
2024-04-10 15:59:46.616 [SimpleAsyncTaskExecutor-921] INFO c.s.e.m.service.component.agentjob.JobExecutor - execute ClusterUpgrade job 2 uuid abeca9c4a402433a877fdf9c9fa05cd0.
2024-04-10 15:59:46.616 [SimpleAsyncTaskExecutor-921] INFO c.s.e.m.service.component.agentjob.JobExecutor - get ClusterUpgrade job 2 uuid abeca9c4a402433a877fdf9c9fa05cd0.
2024-04-10 15:59:46.625 [SimpleAsyncTaskExecutor-921] INFO c.s.e.m.s.component.agentjob.job.UpgradeClusterJob - upgrade cluster job 2 uuid abeca9c4a402433a877fdf9c9fa05cd0 fail.
