flink 任务添加了 elasticsearch 之后 log4j 日志的 jar出现了冲突， 下面是一些报错日志以及解决方式：
1. 提交任务时未成功提交的日志
Setting HBASE_CONF_DIR=/etc/hbase/conf because no HBASE_CONF_DIR was set.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/bigdata/data-process/flink/flink-1.13.6/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/3.1.0.0-78/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
2023-08-30 14:00:59,937 WARN org.apache.flink.yarn.configuration.YarnLogConfigUtil [] - The configuration directory ('/bigdata/data-process/flink/flink-1.13.6/conf') already contains a LOG4J config file.If you want to use logback, then please delete or rename the log configuration file.
2023-08-30 14:01:00,242 INFO org.apache.hadoop.yarn.client.RMProxy [] - Connecting to ResourceManager at nodej01/192.168.71.70:8050
2023-08-30 14:01:00,506 INFO org.apache.hadoop.yarn.client.AHSProxy [] - Connecting to Application History server at nodej02/192.168.71.71:10200
2023-08-30 14:01:00,515 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - No path for the flink jar passed. Using the location of class org.apache.flink.yarn.YarnClusterDescriptor to locate the jar
2023-08-30 14:01:00,816 INFO org.apache.hadoop.conf.Configuration [] - found resource resource-types.xml at file:/etc/hadoop/3.1.0.0-78/0/resource-types.xml
2023-08-30 14:01:00,949 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - The configured JobManager memory is 1600 MB. YARN will allocate 2048 MB to make up an integer multiple of its minimum allocation memory (2048 MB, configured via 'yarn.scheduler.minimum-allocation-mb'). The extra 448 MB may not be used by Flink.
2023-08-30 14:01:00,949 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - The configured TaskManager memory is 1728 MB. YARN will allocate 2048 MB to make up an integer multiple of its minimum allocation memory (2048 MB, configured via 'yarn.scheduler.minimum-allocation-mb'). The extra 320 MB may not be used by Flink.
2023-08-30 14:01:00,949 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Cluster specification: ClusterSpecification{masterMemoryMB=2048, taskManagerMemoryMB=1728, slotsPerTaskManager=1}
2023-08-30 14:01:01,835 WARN org.apache.hadoop.hdfs.shortcircuit.DomainSocketFactory [] - The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.
2023-08-30 14:01:07,671 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Submitting application master application_1687922799600_0950
2023-08-30 14:01:07,749 INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl [] - Submitted application application_1687922799600_0950
2023-08-30 14:01:07,749 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Waiting for the cluster to be allocated
2023-08-30 14:01:07,751 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Deploying cluster, current state ACCEPTED
------------------------------------------------------------
The program finished with the following exception:
org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Could not deploy Yarn job cluster.
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:372)
at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:222)
at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:114)
at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:812)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:246)
at org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1054)
at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1132)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1132)
Caused by: org.apache.flink.client.deployment.ClusterDeploymentException: Could not deploy Yarn job cluster.
at org.apache.flink.yarn.YarnClusterDescriptor.deployJobCluster(YarnClusterDescriptor.java:481)
at org.apache.flink.client.deployment.executors.AbstractJobClusterExecutor.execute(AbstractJobClusterExecutor.java:81)
at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:1956)
at org.apache.flink.client.program.StreamContextEnvironment.executeAsync(StreamContextEnvironment.java:137)
at org.apache.flink.client.program.StreamContextEnvironment.execute(StreamContextEnvironment.java:76)
at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1833)
at com.act.flink.KafkaToEsFromImageFeature.main(KafkaToEsFromImageFeature.java:136)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:355)
... 11 more
Caused by: org.apache.flink.yarn.YarnClusterDescriptor$YarnDeploymentException: The YARN application unexpectedly switched to state FAILED during deployment.
Diagnostics from YARN: Application application_1687922799600_0950 failed 1 times (global limit =2; local limit is =1) due to AM Container for appattempt_1687922799600_0950_000001 exited with exitCode: 1
Failing this attempt.Diagnostics: [2023-08-30 14:01:14.268]Exception from container-launch.
Container id: container_e08_1687922799600_0950_01_000001
Exit code: 1
Exception message: Launch container failed
Shell output: main : command provided 1
main : run as user is nobody
main : requested yarn user is root
Getting exit code file...
Creating script paths...
Writing pid file...
Writing to tmp file /bigdata/hadoop/yarn/local/nmPrivate/application_1687922799600_0950/container_e08_1687922799600_0950_01_000001/container_e08_1687922799600_0950_01_000001.pid.tmp
Writing to cgroup task files...
Creating local dirs...
Launching container...
Getting exit code file...
Creating script paths...
[2023-08-30 14:01:14.290]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
[2023-08-30 14:01:14.293]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
For more detailed output, check the application tracking page: http://nodej01:12088/cluster/app/application_1687922799600_0950 Then click on links to logs of each attempt.
. Failing the application.
If log aggregation is enabled on your cluster, use this command to further investigate the issue:
yarn logs -applicationId application_1687922799600_0950
at org.apache.flink.yarn.YarnClusterDescriptor.startAppMaster(YarnClusterDescriptor.java:1201)
at org.apache.flink.yarn.YarnClusterDescriptor.deployInternal(YarnClusterDescriptor.java:593)
at org.apache.flink.yarn.YarnClusterDescriptor.deployJobCluster(YarnClusterDescriptor.java:474)
... 22 more
2023-08-30 14:01:14,668 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Cancelling deployment from Deployment Failure Hook
2023-08-30 14:01:14,669 INFO org.apache.hadoop.yarn.client.RMProxy [] - Connecting to ResourceManager at nodej01/192.168.71.70:8050
2023-08-30 14:01:14,690 INFO org.apache.hadoop.yarn.client.AHSProxy [] - Connecting to Application History server at nodej02/192.168.71.71:10200
2023-08-30 14:01:14,691 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Killing YARN application
2023-08-30 14:01:14,718 INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl [] - Killed application application_1687922799600_0950
2023-08-30 14:01:14,819 INFO org.apache.flink.yarn.YarnClusterDescriptor [] - Deleting files in hdfs://nodej01:8020/user/root/.flink/application_1687922799600_0950.
2. 拉取 yarn 上的日志
yarn logs -applicationId application_1687922799600_0933
23/08/30 13:48:16 INFO client.RMProxy: Connecting to ResourceManager at node01/192.168.71.70:8050
23/08/30 13:48:17 INFO client.AHSProxy: Connecting to Application History server at node02/192.168.71.71:10200
23/08/30 13:48:18 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
23/08/30 13:48:18 INFO compress.CodecPool: Got brand-new decompressor [.deflate]
Container: container_e08_1687922799600_0933_01_000001 on node01_45454_1693374386766
LogAggregationType: AGGREGATED
======================================================================================
LogType:container-localizer-syslog
LogLastModifiedTime:Wed Aug 30 13:46:26 +0800 2023
LogLength:349
LogContents:
2023-08-30 13:46:18,386 WARN [main] org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer: Localization running as nobody not root
2023-08-30 13:46:18,969 INFO [main] org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer: Disk Validator: yarn.nodemanager.disk-validator is loaded.
End of LogType:container-localizer-syslog
*******************************************************************************************
Container: container_e08_1687922799600_0933_01_000001 on node01_45454_1693374386766
LogAggregationType: AGGREGATED
======================================================================================
LogType:directory.info
LogLastModifiedTime:Wed Aug 30 13:46:26 +0800 2023
LogLength:5525
LogContents:
ls -l:
total 28
-rw-------. 1 nobody hadoop 70 Aug 30 13:46 container_tokens
lrwxrwxrwx. 1 nobody hadoop 164 Aug 30 13:46 flink-conf.yaml -> /bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/18/application_1687922799600_0933-flink-conf.yaml5969087029126036280.tmp
lrwxrwxrwx. 1 nobody hadoop 121 Aug 30 13:46 flink-dist_2.12-1.13.6.jar -> /bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/22/flink-dist_2.12-1.13.6.jar
lrwxrwxrwx. 1 nobody hadoop 135 Aug 30 13:46 flink--jar-with-dependencies.jar -> /bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/21/flink--jar-with-dependencies.jar
lrwxrwxrwx. 1 nobody hadoop 148 Aug 30 13:46 job.graph -> /bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/23/application_1687922799600_09336955381262010602857.tmp
-rwx------. 1 nobody hadoop 17023 Aug 30 13:46 launch_container.sh
drwxr-s---. 2 nobody hadoop 4096 Aug 30 13:46 lib
lrwxrwxrwx. 1 nobody hadoop 111 Aug 30 13:46 log4j.properties -> /bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/16/log4j.properties
drwxr-s---. 10 nobody hadoop 250 Aug 30 13:46 plugins
drwxr-s---. 2 nobody hadoop 6 Aug 30 13:46 tmp
find -L . -maxdepth 5 -ls:
594651 4 drwxr-s--- 5 nobody hadoop 4096 Aug 30 13:46 .
2148617758 0 drwxr-s--- 2 nobody hadoop 6 Aug 30 13:46 ./tmp
594654 20 -rwx------ 1 nobody hadoop 17023 Aug 30 13:46 ./launch_container.sh
594655 4 -rw------- 1 nobody hadoop 70 Aug 30 13:46 ./container_tokens
19327523668 60 -r-x------ 1 nobody nobody 59672 Aug 30 13:46 ./job.graph
15042320357 104048 -r-x------ 1 nobody nobody 106542761 Aug 30 13:46 ./flink-dist_2.12-1.13.6.jar
24629443601 4 -r-x------ 1 nobody nobody 2723 Aug 30 13:46 ./log4j.properties
4300192393 0 drwxr-s--- 10 nobody hadoop 250 Aug 30 13:46 ./plugins
6444649919 0 drwxr-s--- 2 nobody hadoop 55 Aug 30 13:46 ./plugins/metrics-graphite
19332161598 172 -r-x------ 1 nobody nobody 175139 Aug 30 13:46 ./plugins/metrics-graphite/flink-metrics-graphite-1.13.6.jar
8590259050 0 drwxr-s--- 2 nobody hadoop 55 Aug 30 13:46 ./plugins/metrics-influx
6444649911 956 -r-x------ 1 nobody nobody 978710 Aug 30 13:46 ./plugins/metrics-influx/flink-metrics-influxdb-1.13.6.jar
12885353448 0 drwxr-s--- 2 nobody hadoop 57 Aug 30 13:46 ./plugins/metrics-prometheus
2148617735 100 -r-x------ 1 nobody nobody 101123 Aug 30 13:46 ./plugins/metrics-prometheus/flink-metrics-prometheus-1.13.6.jar
15042320386 0 drwxr-s--- 2 nobody hadoop 53 Aug 30 13:46 ./plugins/metrics-statsd
24629443612 12 -r-x------ 1 nobody nobody 11902 Aug 30 13:46 ./plugins/metrics-statsd/flink-metrics-statsd-1.13.6.jar
17186080326 0 drwxr-s--- 2 nobody hadoop 50 Aug 30 13:46 ./plugins/metrics-jmx
15042320383 20 -r-x------ 1 nobody nobody 17973 Aug 30 13:46 ./plugins/metrics-jmx/flink-metrics-jmx-1.13.6.jar
19332201120 0 drwxr-s--- 2 nobody hadoop 130 Aug 30 13:46 ./plugins/external-resource-gpu
2148617741 4 -r-x------ 1 nobody nobody 3189 Aug 30 13:46 ./plugins/external-resource-gpu/gpu-discovery-common.sh
24629443608 4 -r-x------ 1 nobody nobody 1794 Aug 30 13:46 ./plugins/external-resource-gpu/nvidia-gpu-discovery.sh
27920791832 16 -r-x------ 1 nobody nobody 15741 Aug 30 13:46 ./plugins/external-resource-gpu/flink-external-resource-gpu-1.13.6.jar
24629443604 0 drwxr-s--- 2 nobody hadoop 52 Aug 30 13:46 ./plugins/metrics-slf4j
19329304621 12 -r-x------ 1 nobody nobody 10032 Aug 30 13:46 ./plugins/metrics-slf4j/flink-metrics-slf4j-1.13.6.jar
24629443615 0 drwxr-s--- 2 nobody hadoop 54 Aug 30 13:46 ./plugins/metrics-datadog
6444649917 492 -r-x------ 1 nobody nobody 503276 Aug 30 13:46 ./plugins/metrics-datadog/flink-metrics-datadog-1.13.6.jar
10738166835 4 -r-x------ 1 nobody nobody 654 Aug 30 13:46 ./plugins/README.txt
10738166837 4 drwxr-s--- 2 nobody hadoop 4096 Aug 30 13:46 ./lib
27920791821 37720 -r-x------ 1 nobody nobody 38622330 Aug 30 13:46 ./lib/flink-table-blink_2.12-1.13.6.jar
15042320353 204 -r-x------ 1 nobody nobody 208006 Aug 30 13:46 ./lib/log4j-1.2-api-2.17.1.jar
32213151194 148 -r-x------ 1 nobody nobody 148127 Aug 30 13:46 ./lib/flink-json-1.13.6.jar
10738166819 24 -r-x------ 1 nobody nobody 24279 Aug 30 13:46 ./lib/log4j-slf4j-impl-2.17.1.jar
6444649914 92 -r-x------ 1 nobody nobody 92314 Aug 30 13:46 ./lib/flink-csv-1.13.6.jar
27917294447 7532 -r-x------ 1 nobody nobody 7709740 Aug 30 13:46 ./lib/flink-shaded-zookeeper-3.4.14.jar
2148617738 296 -r-x------ 1 nobody nobody 301872 Aug 30 13:46 ./lib/log4j-api-2.17.1.jar
25770817645 34236 -r-x------ 1 nobody nobody 35053606 Aug 30 13:46 ./lib/flink-table_2.12-1.13.6.jar
27920791835 1752 -r-x------ 1 nobody nobody 1790452 Aug 30 13:46 ./lib/log4j-core-2.17.1.jar
32213151191 4 -r-x------ 1 nobody nobody 1042 Aug 30 13:46 ./flink-conf.yaml
10738166824 120580 -r-x------ 1 nobody nobody 123473504 Aug 30 13:46 ./flink--jar-with-dependencies.jar
broken symlinks(find -L . -maxdepth 5 -type l -ls):
End of LogType:directory.info
*******************************************************************************
Container: container_e08_1687922799600_0933_01_000001 on node01_45454_1693374386766
LogAggregationType: AGGREGATED
======================================================================================
LogType:jobmanager.err
LogLastModifiedTime:Wed Aug 30 13:46:26 +0800 2023
LogLength:1163
LogContents:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/13/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/3.1.0.0-78/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.logging.log4j.spi.LoggerContextFactory.isClassLoaderDependent()Z
at org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:49)
at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:46)
at org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:33)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:329)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:349)
at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.<clinit>(ClusterEntrypoint.java:108)
End of LogType:jobmanager.err
*******************************************************************************
End of LogType:jobmanager.out
*******************************************************************************
Container: container_e08_1687922799600_0933_01_000001 on node01_45454_1693374386766
LogAggregationType: AGGREGATED
======================================================================================
LogType:launch_container.sh
LogLastModifiedTime:Wed Aug 30 13:46:26 +0800 2023
LogLength:17023
LogContents:
#!/bin/bash
set -o pipefail -e
export PRELAUNCH_OUT="/bigdata/hadoop/yarn/log/application_1687922799600_0933/container_e08_1687922799600_0933_01_000001/prelaunch.out"
exec >"${PRELAUNCH_OUT}"
export PRELAUNCH_ERR="/bigdata/hadoop/yarn/log/application_1687922799600_0933/container_e08_1687922799600_0933_01_000001/prelaunch.err"
exec 2>"${PRELAUNCH_ERR}"
echo "Setting up env variables"
export JAVA_HOME=${JAVA_HOME:-"/usr/local/jdk1.8.0_191"}
export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/usr/hdp/3.1.0.0-78/hadoop/conf"}
export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-"/usr/hdp/3.1.0.0-78/hadoop-yarn"}
export HADOOP_HOME=${HADOOP_HOME:-"/usr/hdp/3.1.0.0-78/hadoop"}
export PATH=${PATH:-"/usr/sbin:/sbin:/usr/lib/ambari-server/*:/usr/local/jdk1.8.0_191/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin:/var/lib/ambari-agent"}
export LANG=${LANG:-"en_US.UTF-8"}
export HADOOP_TOKEN_FILE_LOCATION="/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/container_e08_1687922799600_0933_01_000001/container_tokens"
export CONTAINER_ID="container_e08_1687922799600_0933_01_000001"
export NM_PORT="45454"
export NM_HOST="node01"
export NM_HTTP_PORT="8042"
export LOCAL_DIRS="/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933"
export LOCAL_USER_DIRS="/bigdata/hadoop/yarn/local/usercache/root/"
export LOG_DIRS="/bigdata/hadoop/yarn/log/application_1687922799600_0933/container_e08_1687922799600_0933_01_000001"
export USER="root"
export LOGNAME="root"
export HOME="/home/"
export PWD="/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/container_e08_1687922799600_0933_01_000001"
export JVM_PID="$$"
export MALLOC_ARENA_MAX="4"
export NM_AUX_SERVICE_timeline_collector=""
export NM_AUX_SERVICE_mapreduce_shuffle="AAA0+gAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
export NM_AUX_SERVICE_spark2_shuffle=""
export _CLIENT_HOME_DIR="hdfs://node01:8020/user/root"
export APP_SUBMIT_TIME_ENV="1693374374141"
export _APP_ID="application_1687922799600_0933"
export HADOOP_USER_NAME="root"
export TIMELINE_FLOW_NAME_TAG="flink-dwd_picture_features_to_es"
export TIMELINE_FLOW_VERSION_TAG="1"
export APPLICATION_WEB_PROXY_BASE="/proxy/application_1687922799600_0933"
export _CLIENT_SHIP_FILES="YarnLocalResourceDescriptor{key=log4j.properties, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/log4j.properties, size=2723, modificationTime=1693374368642, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=lib/log4j-1.2-api-2.17.1.jar, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/lib/log4j-1.2-api-2.17.1.jar, size=208006, modificationTime=1693374368718, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=lib/log4j-api-2.17.1.jar, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/lib/log4j-api-2.17.1.jar, size=301872, modificationTime=1693374368750, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=lib/log4j-core-2.17.1.jar, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/lib/log4j-core-2.17.1.jar, size=1790452, modificationTime=1693374368813, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=lib/log4j-slf4j-impl-2.17.1.jar, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/lib/log4j-slf4j-impl-2.17.1.jar, size=24279, modificationTime=1693374368849, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=lib/flink-json-1.13.6.jar, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/lib/flink-json-1.13.6.jar, size=148127, modificationTime=1693374368877, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=lib/flink-shaded-zookeeper-3.4.14.jar, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/lib/flink-shaded-zookeeper-3.4.14.jar, size=7709740, modificationTime=1693374369017, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=lib/flink-table_2.12-1.13.6.jar, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/lib/flink-table_2.12-1.13.6.jar, size=35053606, modificationTime=1693374369550, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=lib/flink-csv-1.13.6.jar, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/lib/flink-csv-1.13.6.jar, size=92314, modificationTime=1693374369579, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=lib/flink-table-blink_2.12-1.13.6.jar, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/lib/flink-table-blink_2.12-1.13.6.jar, size=38622330, modificationTime=1693374370138, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=plugins/external-resource-gpu/flink-external-resource-gpu-1.13.6.jar, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/plugins/external-resource-gpu/flink-external-resource-gpu-1.13.6.jar, size=15741, modificationTime=1693374370208, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=plugins/external-resource-gpu/gpu-discovery-common.sh, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/plugins/external-resource-gpu/gpu-discovery-common.sh, size=3189, modificationTime=1693374370243, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=plugins/external-resource-gpu/nvidia-gpu-discovery.sh, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/plugins/external-resource-gpu/nvidia-gpu-discovery.sh, size=1794, modificationTime=1693374370267, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=plugins/metrics-datadog/flink-metrics-datadog-1.13.6.jar, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/plugins/metrics-datadog/flink-metrics-datadog-1.13.6.jar, size=503276, modificationTime=1693374370298, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=plugins/metrics-graphite/flink-metrics-graphite-1.13.6.jar, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/plugins/metrics-graphite/flink-metrics-graphite-1.13.6.jar, size=175139, modificationTime=1693374370333, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=plugins/metrics-influx/flink-metrics-influxdb-1.13.6.jar, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/plugins/metrics-influx/flink-metrics-influxdb-1.13.6.jar, size=978710, modificationTime=1693374370445, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=plugins/metrics-jmx/flink-metrics-jmx-1.13.6.jar, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/plugins/metrics-jmx/flink-metrics-jmx-1.13.6.jar, size=17973, modificationTime=1693374370470, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=plugins/metrics-prometheus/flink-metrics-prometheus-1.13.6.jar, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/plugins/metrics-prometheus/flink-metrics-prometheus-1.13.6.jar, size=101123, modificationTime=1693374370490, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=plugins/metrics-slf4j/flink-metrics-slf4j-1.13.6.jar, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/plugins/metrics-slf4j/flink-metrics-slf4j-1.13.6.jar, size=10032, modificationTime=1693374370516, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=plugins/metrics-statsd/flink-metrics-statsd-1.13.6.jar, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/plugins/metrics-statsd/flink-metrics-statsd-1.13.6.jar, size=11902, modificationTime=1693374370565, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=plugins/README.txt, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/plugins/README.txt, size=654, modificationTime=1693374370587, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=flink--jar-with-dependencies.jar, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/flink--jar-with-dependencies.jar, size=123473504, modificationTime=1693374372495, visibility=APPLICATION, type=FILE};YarnLocalResourceDescriptor{key=flink-conf.yaml, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/application_1687922799600_0933-flink-conf.yaml5969087029126036280.tmp, size=1042, modificationTime=1693374374089, visibility=APPLICATION, type=FILE}"
export CLASSPATH=":flink--jar-with-dependencies.jar:lib/flink-csv-1.13.6.jar:lib/flink-json-1.13.6.jar:lib/flink-shaded-zookeeper-3.4.14.jar:lib/flink-table-blink_2.12-1.13.6.jar:lib/flink-table_2.12-1.13.6.jar:lib/log4j-1.2-api-2.17.1.jar:lib/log4j-api-2.17.1.jar:lib/log4j-core-2.17.1.jar:lib/log4j-slf4j-impl-2.17.1.jar:flink-dist_2.12-1.13.6.jar:job.graph:flink-conf.yaml::$HADOOP_CONF_DIR:/usr/hdp/3.1.0.0-78/hadoop/*:/usr/hdp/3.1.0.0-78/hadoop/lib/*:/usr/hdp/current/hadoop-hdfs-client/*:/usr/hdp/current/hadoop-hdfs-client/lib/*:/usr/hdp/current/hadoop-yarn-client/*:/usr/hdp/current/hadoop-yarn-client/lib/*"
export _FLINK_YARN_FILES="hdfs://node01:8020/user/root/.flink/application_1687922799600_0933"
export _FLINK_CLASSPATH=":flink--jar-with-dependencies.jar:lib/flink-csv-1.13.6.jar:lib/flink-json-1.13.6.jar:lib/flink-shaded-zookeeper-3.4.14.jar:lib/flink-table-blink_2.12-1.13.6.jar:lib/flink-table_2.12-1.13.6.jar:lib/log4j-1.2-api-2.17.1.jar:lib/log4j-api-2.17.1.jar:lib/log4j-core-2.17.1.jar:lib/log4j-slf4j-impl-2.17.1.jar:flink-dist_2.12-1.13.6.jar:job.graph:flink-conf.yaml:"
export TIMELINE_FLOW_RUN_ID_TAG="1693374374196"
export _FLINK_DIST_JAR="YarnLocalResourceDescriptor{key=flink-dist_2.12-1.13.6.jar, path=hdfs://node01:8020/user/root/.flink/application_1687922799600_0933/flink-dist_2.12-1.13.6.jar, size=106542761, modificationTime=1693374374000, visibility=APPLICATION, type=FILE}"
echo "Setting up job resources"
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/23/application_1687922799600_09336955381262010602857.tmp" "job.graph"
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/22/flink-dist_2.12-1.13.6.jar" "flink-dist_2.12-1.13.6.jar"
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/16/log4j.properties" "log4j.properties"
mkdir -p plugins/metrics-graphite
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/15/flink-metrics-graphite-1.13.6.jar" "plugins/metrics-graphite/flink-metrics-graphite-1.13.6.jar"
mkdir -p plugins/metrics-influx
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/12/flink-metrics-influxdb-1.13.6.jar" "plugins/metrics-influx/flink-metrics-influxdb-1.13.6.jar"
mkdir -p lib
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/10/flink-table-blink_2.12-1.13.6.jar" "lib/flink-table-blink_2.12-1.13.6.jar"
mkdir -p plugins/metrics-prometheus
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/11/flink-metrics-prometheus-1.13.6.jar" "plugins/metrics-prometheus/flink-metrics-prometheus-1.13.6.jar"
mkdir -p plugins/metrics-statsd
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/32/flink-metrics-statsd-1.13.6.jar" "plugins/metrics-statsd/flink-metrics-statsd-1.13.6.jar"
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/18/application_1687922799600_0933-flink-conf.yaml5969087029126036280.tmp" "flink-conf.yaml"
mkdir -p plugins/metrics-jmx
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/30/flink-metrics-jmx-1.13.6.jar" "plugins/metrics-jmx/flink-metrics-jmx-1.13.6.jar"
mkdir -p lib
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/14/log4j-1.2-api-2.17.1.jar" "lib/log4j-1.2-api-2.17.1.jar"
mkdir -p lib
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/26/flink-json-1.13.6.jar" "lib/flink-json-1.13.6.jar"
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/21/flink--jar-with-dependencies.jar" "flink--jar-with-dependencies.jar"
mkdir -p lib
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/13/log4j-slf4j-impl-2.17.1.jar" "lib/log4j-slf4j-impl-2.17.1.jar"
mkdir -p lib
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/20/flink-csv-1.13.6.jar" "lib/flink-csv-1.13.6.jar"
mkdir -p lib
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/34/flink-shaded-zookeeper-3.4.14.jar" "lib/flink-shaded-zookeeper-3.4.14.jar"
mkdir -p plugins/external-resource-gpu
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/27/gpu-discovery-common.sh" "plugins/external-resource-gpu/gpu-discovery-common.sh"
mkdir -p plugins/metrics-slf4j
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/31/flink-metrics-slf4j-1.13.6.jar" "plugins/metrics-slf4j/flink-metrics-slf4j-1.13.6.jar"
mkdir -p plugins/external-resource-gpu
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/24/nvidia-gpu-discovery.sh" "plugins/external-resource-gpu/nvidia-gpu-discovery.sh"
mkdir -p plugins/metrics-datadog
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/28/flink-metrics-datadog-1.13.6.jar" "plugins/metrics-datadog/flink-metrics-datadog-1.13.6.jar"
mkdir -p plugins
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/29/README.txt" "plugins/README.txt"
mkdir -p lib
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/19/log4j-api-2.17.1.jar" "lib/log4j-api-2.17.1.jar"
mkdir -p lib
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/33/flink-table_2.12-1.13.6.jar" "lib/flink-table_2.12-1.13.6.jar"
mkdir -p plugins/external-resource-gpu
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/17/flink-external-resource-gpu-1.13.6.jar" "plugins/external-resource-gpu/flink-external-resource-gpu-1.13.6.jar"
mkdir -p lib
ln -sf "/bigdata/hadoop/yarn/local/usercache/root/appcache/application_1687922799600_0933/filecache/25/log4j-core-2.17.1.jar" "lib/log4j-core-2.17.1.jar"
echo "Copying debugging information"
# Creating copy of launch script
cp "launch_container.sh" "/bigdata/hadoop/yarn/log/application_1687922799600_0933/container_e08_1687922799600_0933_01_000001/launch_container.sh"
chmod 640 "/bigdata/hadoop/yarn/log/application_1687922799600_0933/container_e08_1687922799600_0933_01_000001/launch_container.sh"
# Determining directory contents
echo "ls -l:" 1>"/bigdata/hadoop/yarn/log/application_1687922799600_0933/container_e08_1687922799600_0933_01_000001/directory.info"
ls -l 1>>"/bigdata/hadoop/yarn/log/application_1687922799600_0933/container_e08_1687922799600_0933_01_000001/directory.info"
echo "find -L . -maxdepth 5 -ls:" 1>>"/bigdata/hadoop/yarn/log/application_1687922799600_0933/container_e08_1687922799600_0933_01_000001/directory.info"
find -L . -maxdepth 5 -ls 1>>"/bigdata/hadoop/yarn/log/application_1687922799600_0933/container_e08_1687922799600_0933_01_000001/directory.info"
echo "broken symlinks(find -L . -maxdepth 5 -type l -ls):" 1>>"/bigdata/hadoop/yarn/log/application_1687922799600_0933/container_e08_1687922799600_0933_01_000001/directory.info"
find -L . -maxdepth 5 -type l -ls 1>>"/bigdata/hadoop/yarn/log/application_1687922799600_0933/container_e08_1687922799600_0933_01_000001/directory.info"
echo "Launching container"
exec /bin/bash -c "$JAVA_HOME/bin/java -Xmx1073741824 -Xms1073741824 -XX:MaxMetaspaceSize=268435456 -Dlog.file="/bigdata/hadoop/yarn/log/application_1687922799600_0933/container_e08_1687922799600_0933_01_000001/jobmanager.log" -Dlog4j.configuration=file:log4j.properties -Dlog4j.configurationFile=file:log4j.properties org.apache.flink.yarn.entrypoint.YarnJobClusterEntrypoint -D jobmanager.memory.off-heap.size=134217728b -D jobmanager.memory.jvm-overhead.min=201326592b -D jobmanager.memory.jvm-metaspace.size=268435456b -D jobmanager.memory.heap.size=1073741824b -D jobmanager.memory.jvm-overhead.max=201326592b 1> /bigdata/hadoop/yarn/log/application_1687922799600_0933/container_e08_1687922799600_0933_01_000001/jobmanager.out 2> /bigdata/hadoop/yarn/log/application_1687922799600_0933/container_e08_1687922799600_0933_01_000001/jobmanager.err"
End of LogType:launch_container.sh
************************************************************************************
End of LogType:prelaunch.err
******************************************************************************
Container: container_e08_1687922799600_0933_01_000001 on node01_45454_1693374386766
LogAggregationType: AGGREGATED
======================================================================================
LogType:prelaunch.out
LogLastModifiedTime:Wed Aug 30 13:46:26 +0800 2023
LogLength:100
LogContents:
Setting up env variables
Setting up job resources
Copying debugging information
Launching container
End of LogType:prelaunch.out
******************************************************************************
分析出是 jar包异常后， 打印依赖树
mvn dependency:tree -Dverbose
[INFO] ---------------------< com.hnbian:flink >---------------------
[INFO] Building flink [4/6]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:3.2.0:tree (default-cli) @ flink ---
[INFO] com.hnbian.:flink:jar:
[INFO] +- org.apache.httpcomponents:httpclient:jar:4.5.13:compile
[INFO] | +- org.apache.httpcomponents:httpcore:jar:4.4.13:compile
[INFO] | +- commons-logging:commons-logging:jar:1.2:compile
[INFO] | \- commons-codec:commons-codec:jar:1.11:compile
[INFO] +- org.apache.flink:flink-connector-elasticsearch7_2.12:jar:1.13.6:compile
[INFO] | +- org.apache.flink:flink-connector-elasticsearch-base_2.12:jar:1.13.6:compile
[INFO] | +- org.elasticsearch.client:elasticsearch-rest-high-level-client:jar:7.5.1:compile
[INFO] | | +- org.elasticsearch:elasticsearch:jar:7.5.1:compile
[INFO] | | | +- org.elasticsearch:elasticsearch-core:jar:7.5.1:compile
[INFO] | | | +- org.elasticsearch:elasticsearch-secure-sm:jar:7.5.1:compile
[INFO] | | | +- org.elasticsearch:elasticsearch-x-content:jar:7.5.1:compile
[INFO] | | | | +- org.yaml:snakeyaml:jar:1.17:compile
[INFO] | | | | +- com.fasterxml.jackson.dataformat:jackson-dataformat-smile:jar:2.8.11:compile
[INFO] | | | | +- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.8.11:compile
[INFO] | | | | \- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.8.11:compile
[INFO] | | | +- org.elasticsearch:elasticsearch-geo:jar:7.5.1:compile
[INFO] | | | +- org.apache.lucene:lucene-core:jar:8.3.0:compile
[INFO] | | | +- org.apache.lucene:lucene-analyzers-common:jar:8.3.0:compile
[INFO] | | | +- org.apache.lucene:lucene-backward-codecs:jar:8.3.0:compile
[INFO] | | | +- org.apache.lucene:lucene-grouping:jar:8.3.0:compile
[INFO] | | | +- org.apache.lucene:lucene-highlighter:jar:8.3.0:compile
[INFO] | | | +- org.apache.lucene:lucene-join:jar:8.3.0:compile
[INFO] | | | +- org.apache.lucene:lucene-memory:jar:8.3.0:compile
[INFO] | | | +- org.apache.lucene:lucene-misc:jar:8.3.0:compile
[INFO] | | | +- org.apache.lucene:lucene-queries:jar:8.3.0:compile
[INFO] | | | +- org.apache.lucene:lucene-queryparser:jar:8.3.0:compile
[INFO] | | | +- org.apache.lucene:lucene-sandbox:jar:8.3.0:compile
[INFO] | | | +- org.apache.lucene:lucene-spatial:jar:8.3.0:compile
[INFO] | | | +- org.apache.lucene:lucene-spatial-extras:jar:8.3.0:compile
[INFO] | | | +- org.apache.lucene:lucene-spatial3d:jar:8.3.0:compile
[INFO] | | | +- org.apache.lucene:lucene-suggest:jar:8.3.0:compile
[INFO] | | | +- org.elasticsearch:elasticsearch-cli:jar:7.5.1:compile
[INFO] | | | | \- net.sf.jopt-simple:jopt-simple:jar:5.0.2:compile
[INFO] | | | +- com.carrotsearch:hppc:jar:0.8.1:compile
[INFO] | | | +- com.tdunning:t-digest:jar:3.2:compile
[INFO] | | | +- org.hdrhistogram:HdrHistogram:jar:2.1.9:compile
[INFO] | | | +- org.apache.logging.log4j:log4j-api:jar:2.11.1:compile #log4j
[INFO] | | | \- org.elasticsearch:jna:jar:4.5.1:compile
[INFO] | | +- org.elasticsearch.client:elasticsearch-rest-client:jar:7.5.1:compile
[INFO] | | | +- org.apache.httpcomponents:httpasyncclient:jar:4.1.4:compile
[INFO] | | | \- org.apache.httpcomponents:httpcore-nio:jar:4.4.12:compile
[INFO] | | +- org.elasticsearch.plugin:mapper-extras-client:jar:7.5.1:compile
[INFO] | | +- org.elasticsearch.plugin:parent-join-client:jar:7.5.1:compile
[INFO] | | +- org.elasticsearch.plugin:aggs-matrix-stats-client:jar:7.5.1:compile
[INFO] | | +- org.elasticsearch.plugin:rank-eval-client:jar:7.5.1:compile
[INFO] | | \- org.elasticsearch.plugin:lang-mustache-client:jar:7.5.1:compile
[INFO] | | \- com.github.spullara.mustache.java:compiler:jar:0.9.6:compile
[INFO] | \- org.apache.flink:force-shading:jar:1.13.6:compile
[INFO] +- org.apache.flink:flink-core:jar:1.13.6:compile
[INFO] | +- org.apache.flink:flink-annotations:jar:1.13.6:compile
[INFO] | +- org.apache.flink:flink-metrics-core:jar:1.13.6:compile
[INFO] | +- org.apache.flink:flink-shaded-asm-7:jar:7.1-13.0:compile
[INFO] | +- org.apache.commons:commons-lang3:jar:3.3.2:compile
[INFO] | +- com.esotericsoftware.kryo:kryo:jar:2.24.0:compile
[INFO] | | +- com.esotericsoftware.minlog:minlog:jar:1.2:compile
[INFO] | | \- org.objenesis:objenesis:jar:2.1:compile
[INFO] | +- commons-collections:commons-collections:jar:3.2.2:compile
[INFO] | +- org.apache.commons:commons-compress:jar:1.21:compile
[INFO] | +- org.apache.flink:flink-shaded-guava:jar:18.0-13.0:compile
[INFO] | +- org.slf4j:slf4j-api:jar:1.7.15:compile
[INFO] | \- com.google.code.findbugs:jsr305:jar:1.3.9:compile
[INFO] +- org.apache.flink:flink-streaming-java_2.12:jar:1.13.6:compile
[INFO] | +- org.apache.flink:flink-file-sink-common:jar:1.13.6:compile
[INFO] | +- org.apache.flink:flink-runtime_2.12:jar:1.13.6:compile
[INFO] | | +- org.apache.flink:flink-queryable-state-client-java:jar:1.13.6:compile
[INFO] | | +- org.apache.flink:flink-hadoop-fs:jar:1.13.6:compile
[INFO] | | +- org.apache.flink:flink-shaded-netty:jar:4.1.49.Final-13.0:compile
[INFO] | | +- org.apache.flink:flink-shaded-jackson:jar:2.12.1-13.0:compile
[INFO] | | +- org.apache.flink:flink-shaded-zookeeper-3:jar:3.4.14-13.0:compile
[INFO] | | +- org.javassist:javassist:jar:3.24.0-GA:compile
[INFO] | | +- org.scala-lang:scala-library:jar:2.12.7:compile
[INFO] | | +- com.typesafe.akka:akka-actor_2.12:jar:2.5.21:compile
[INFO] | | | +- com.typesafe:config:jar:1.3.3:compile
[INFO] | | | \- org.scala-lang.modules:scala-java8-compat_2.12:jar:0.8.0:compile
[INFO] | | +- com.typesafe.akka:akka-stream_2.12:jar:2.5.21:compile
[INFO] | | | +- org.reactivestreams:reactive-streams:jar:1.0.2:compile
[INFO] | | | \- com.typesafe:ssl-config-core_2.12:jar:0.3.7:compile
[INFO] | | | \- org.scala-lang.modules:scala-parser-combinators_2.12:jar:1.1.1:compile
[INFO] | | +- com.typesafe.akka:akka-protobuf_2.12:jar:2.5.21:compile
[INFO] | | +- com.typesafe.akka:akka-slf4j_2.12:jar:2.5.21:compile
[INFO] | | +- org.clapper:grizzled-slf4j_2.12:jar:1.3.2:compile
[INFO] | | +- com.github.scopt:scopt_2.12:jar:3.5.0:compile
[INFO] | | +- com.twitter:chill_2.12:jar:0.7.6:compile
[INFO] | | | \- com.twitter:chill-java:jar:0.7.6:compile
[INFO] | | \- org.lz4:lz4-java:jar:1.6.0:compile
[INFO] | +- org.apache.flink:flink-java:jar:1.13.6:compile
[INFO] | \- org.apache.commons:commons-math3:jar:3.5:compile
[INFO] +- org.apache.flink:flink-connector-kafka_2.12:jar:1.13.6:compile
[INFO] | +- org.apache.kafka:kafka-clients:jar:2.4.1:compile
[INFO] | | \- com.github.luben:zstd-jni:jar:1.4.3-1:compile
[INFO] | \- org.apache.flink:flink-connector-base:jar:1.13.6:compile
[INFO] +- org.apache.flink:flink-json:jar:1.13.6:compile
[INFO] +- org.apache.flink:flink-avro:jar:1.13.6:compile
[INFO] +- org.apache.flink:flink-connector-jdbc_2.12:jar:1.13.6:compile
[INFO] +- org.apache.flink:flink-table-api-java-bridge_2.12:jar:1.13.6:compile
[INFO] | \- org.apache.flink:flink-table-api-java:jar:1.13.6:compile
[INFO] | \- org.apache.flink:flink-table-common:jar:1.13.6:compile
[INFO] | \- org.apache.flink:flink-connector-files:jar:1.13.6:compile
[INFO] +- org.apache.flink:flink-parquet_2.12:jar:1.13.6:compile
[INFO] +- mysql:mysql-connector-java:jar:8.0.26:compile
[INFO] | \- com.google.protobuf:protobuf-java:jar:3.11.4:compile
[INFO] +- org.apache.flink:flink-clients_2.12:jar:1.13.6:compile
[INFO] | +- org.apache.flink:flink-optimizer_2.12:jar:1.13.6:compile
[INFO] | \- commons-cli:commons-cli:jar:1.3.1:compile
[INFO] +- joda-time:joda-time:jar:2.9.6:compile
[INFO] +- redis.clients:jedis:jar:2.9.0:compile
[INFO] | \- org.apache.commons:commons-pool2:jar:2.4.2:compile
[INFO] +- org.apache.parquet:parquet-common:jar:1.11.0:compile
[INFO] | +- org.apache.parquet:parquet-format-structures:jar:1.11.0:compile
[INFO] | | \- javax.annotation:javax.annotation-api:jar:1.3.2:compile
[INFO] | \- org.apache.yetus:audience-annotations:jar:0.11.0:compile
[INFO] +- org.apache.avro:avro:jar:1.8.2:compile
[INFO] | +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
[INFO] | +- com.thoughtworks.paranamer:paranamer:jar:2.7:compile
[INFO] | +- org.xerial.snappy:snappy-java:jar:1.1.1.3:compile
[INFO] | \- org.tukaani:xz:jar:1.5:compile
[INFO] +- org.apache.parquet:parquet-avro:jar:1.11.0:compile
[INFO] | \- org.apache.parquet:parquet-column:jar:1.11.0:compile
[INFO] | \- org.apache.parquet:parquet-encoding:jar:1.11.0:compile
[INFO] +- org.apache.parquet:parquet-hadoop:jar:1.11.1:compile
[INFO] | +- org.apache.parquet:parquet-jackson:jar:1.11.1:compile
[INFO] | \- commons-pool:commons-pool:jar:1.6:compile
[INFO] +- com.alibaba:fastjson:jar:2.0.6:compile
[INFO] | \- com.alibaba.fastjson2:fastjson2-extension:jar:2.0.6:compile
[INFO] | \- com.alibaba.fastjson2:fastjson2:jar:2.0.6:compile
[INFO] +- commons-lang:commons-lang:jar:2.6:compile
[INFO] +- org.apache.hadoop:hadoop-common:jar:3.1.0:provided
[INFO] | +- com.google.guava:guava:jar:11.0.2:provided
[INFO] | +- commons-io:commons-io:jar:2.5:compile
[INFO] | +- commons-net:commons-net:jar:3.6:provided
[INFO] | +- javax.servlet:javax.servlet-api:jar:3.1.0:provided
[INFO] | +- org.eclipse.jetty:jetty-server:jar:9.3.19.v20170502:provided
[INFO] | | +- org.eclipse.jetty:jetty-http:jar:9.3.19.v20170502:provided
[INFO] | | \- org.eclipse.jetty:jetty-io:jar:9.3.19.v20170502:provided
[INFO] | +- org.eclipse.jetty:jetty-util:jar:9.3.19.v20170502:provided
[INFO] | +- org.eclipse.jetty:jetty-servlet:jar:9.3.19.v20170502:provided
[INFO] | | \- org.eclipse.jetty:jetty-security:jar:9.3.19.v20170502:provided
[INFO] | +- org.eclipse.jetty:jetty-webapp:jar:9.3.19.v20170502:provided
[INFO] | | \- org.eclipse.jetty:jetty-xml:jar:9.3.19.v20170502:provided
[INFO] | +- javax.servlet.jsp:jsp-api:jar:2.1:provided
[INFO] | +- com.sun.jersey:jersey-core:jar:1.19:provided
[INFO] | | \- javax.ws.rs:jsr311-api:jar:1.1.1:provided
[INFO] | +- com.sun.jersey:jersey-servlet:jar:1.19:provided
[INFO] | +- com.sun.jersey:jersey-json:jar:1.19:provided
[INFO] | | +- org.codehaus.jettison:jettison:jar:1.1:provided
[INFO] | | +- com.sun.xml.bind:jaxb-impl:jar:2.2.3-1:provided
[INFO] | | +- org.codehaus.jackson:jackson-jaxrs:jar:1.9.2:provided
[INFO] | | \- org.codehaus.jackson:jackson-xc:jar:1.9.2:provided
[INFO] | +- com.sun.jersey:jersey-server:jar:1.19:provided
[INFO] | +- log4j:log4j:jar:1.2.17:provided # log4j
[INFO] | +- commons-beanutils:commons-beanutils:jar:1.9.3:provided
[INFO] | +- org.apache.commons:commons-configuration2:jar:2.1.1:provided
[INFO] | +- com.google.re2j:re2j:jar:1.1:provided
[INFO] | +- com.google.code.gson:gson:jar:2.2.4:provided
[INFO] | +- org.apache.hadoop:hadoop-auth:jar:3.1.0:provided
[INFO] | | +- com.nimbusds:nimbus-jose-jwt:jar:4.41.1:provided
[INFO] | | | \- com.github.stephenc.jcip:jcip-annotations:jar:1.0-1:provided
[INFO] | | +- net.minidev:json-smart:jar:2.3:provided
[INFO] | | | \- net.minidev:accessors-smart:jar:1.2:provided
[INFO] | | | \- org.ow2.asm:asm:jar:5.0.4:provided
[INFO] | | \- org.apache.curator:curator-framework:jar:2.12.0:provided
[INFO] | +- com.jcraft:jsch:jar:0.1.54:provided
[INFO] | +- org.apache.curator:curator-client:jar:2.12.0:provided
[INFO] | +- org.apache.curator:curator-recipes:jar:2.12.0:provided
[INFO] | +- org.apache.htrace:htrace-core4:jar:4.1.0-incubating:provided
[INFO] | +- org.apache.zookeeper:zookeeper:jar:3.4.9:provided
[INFO] | | \- jline:jline:jar:0.9.94:provided
[INFO] | +- org.apache.kerby:kerb-simplekdc:jar:1.0.1:provided
[INFO] | | +- org.apache.kerby:kerb-client:jar:1.0.1:provided
[INFO] | | | +- org.apache.kerby:kerby-config:jar:1.0.1:provided
[INFO] | | | +- org.apache.kerby:kerb-core:jar:1.0.1:provided
[INFO] | | | | \- org.apache.kerby:kerby-pkix:jar:1.0.1:provided
[INFO] | | | | +- org.apache.kerby:kerby-asn1:jar:1.0.1:provided
[INFO] | | | | \- org.apache.kerby:kerby-util:jar:1.0.1:provided
[INFO] | | | +- org.apache.kerby:kerb-common:jar:1.0.1:provided
[INFO] | | | | \- org.apache.kerby:kerb-crypto:jar:1.0.1:provided
[INFO] | | | +- org.apache.kerby:kerb-util:jar:1.0.1:provided
[INFO] | | | \- org.apache.kerby:token-provider:jar:1.0.1:provided
[INFO] | | \- org.apache.kerby:kerb-admin:jar:1.0.1:provided
[INFO] | | +- org.apache.kerby:kerb-server:jar:1.0.1:provided
[INFO] | | | \- org.apache.kerby:kerb-identity:jar:1.0.1:provided
[INFO] | | \- org.apache.kerby:kerby-xdr:jar:1.0.1:provided
[INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.7.8:provided
[INFO] | | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.7.0:provided
[INFO] | | \- com.fasterxml.jackson.core:jackson-core:jar:2.7.8:compile
[INFO] | +- org.codehaus.woodstox:stax2-api:jar:3.1.4:provided
[INFO] | \- com.fasterxml.woodstox:woodstox-core:jar:5.0.3:provided
[INFO] +- org.apache.hadoop:hadoop-annotations:jar:3.1.0:compile
[INFO] +- org.apache.hadoop:hadoop-hdfs:jar:3.1.0:provided
[INFO] | +- org.eclipse.jetty:jetty-util-ajax:jar:9.3.19.v20170502:provided
[INFO] | +- commons-daemon:commons-daemon:jar:1.0.13:provided
[INFO] | +- io.netty:netty:jar:3.10.5.Final:provided
[INFO] | +- io.netty:netty-all:jar:4.0.52.Final:provided
[INFO] | \- org.fusesource.leveldbjni:leveldbjni-all:jar:1.8:provided
[INFO] \- org.apache.hadoop:hadoop-client:jar:3.1.0:provided
[INFO] +- org.apache.hadoop:hadoop-hdfs-client:jar:3.1.0:provided
[INFO] | \- com.squareup.okhttp:okhttp:jar:2.7.5:provided
[INFO] | \- com.squareup.okio:okio:jar:1.6.0:provided
[INFO] +- org.apache.hadoop:hadoop-yarn-api:jar:3.1.0:provided
[INFO] | \- javax.xml.bind:jaxb-api:jar:2.2.11:provided
[INFO] +- org.apache.hadoop:hadoop-yarn-client:jar:3.1.0:provided
[INFO] +- org.apache.hadoop:hadoop-mapreduce-client-core:jar:3.1.0:provided
[INFO] | \- org.apache.hadoop:hadoop-yarn-common:jar:3.1.0:provided
[INFO] | +- com.sun.jersey:jersey-client:jar:1.19:provided
[INFO] | +- com.fasterxml.jackson.module:jackson-module-jaxb-annotations:jar:2.7.8:provided
[INFO] | \- com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:jar:2.7.8:provided
[INFO] | \- com.fasterxml.jackson.jaxrs:jackson-jaxrs-base:jar:2.7.8:provided
[INFO] \- org.apache.hadoop:hadoop-mapreduce-client-jobclient:jar:3.1.0:provided
[INFO] \- org.apache.hadoop:hadoop-mapreduce-client-common:jar:3.1.0:provided
[INFO]3. 找到冲突包，并排除
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-elasticsearch7_2.12</artifactId>
<version>${flink.version}</version>
<exclusions>
<!-- Exclude log4j-api -->
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
</exclusion>
</exclusions>
</dependency>4. 重新打包部署， 解决问题
完美执行
