Exception in thread "main" java.

2019-08-10  本文已影响0人  三叉戟渔民
场景:

执行spark-submit --queue=root.XXXX --class com.OrderProfitAgg kafkaspark.jar 时候抛出异常

pom.xml如下:

<dependencies>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-streaming_2.11</artifactId>
            <version>2.1.0</version>
        </dependency>

        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_2.10</artifactId>
            <version>2.1.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-streaming-kafka-0-8_2.11</artifactId>
            <version>2.1.0</version>
        </dependency>
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.19</version>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.38</version>
        </dependency>
        <dependency>
            <groupId>commons-dbcp</groupId>
            <artifactId>commons-dbcp</artifactId>
            <version>1.4</version>
        </dependency>
原因:

类org.apache.spark.Logging只在spark 1.5.2及更早得版本才有,高于1.5.2得版本不支持。

解决方法:

下载:spark-core_2.11-1.5.2.logging.jar
然后把spark-core_2.11-1.5.2.logging.jar复制到$SPARK_HOME/jars目录下重新执行spark-submit即可。

上一篇下一篇

猜你喜欢

热点阅读