Maven
新建一个Maven项目，开开心心地写完scala程序，在本地调试没什么问题，要打包部署在spark集群上运行的时候却出错了，说找不到主类
java.lang.ClassNotFoundException: neu.WordCount
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.spark.util.Utils$.classForName(Utils.scala:229)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:700)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)打开使用IDE打好的Jar包，发现里面没有一个.class文件，
原来是使用IDE默认的方式打包Maven项目，会打不进去
解决无法打包.class文件
使用插件scala-maven-plugin
<plugin>
<groupId>net.alchim31.mavengroupId>
<artifactId>scala-maven-pluginartifactId>
<version>3.2.2version>
<executions>
<execution>
<id>eclipse-add-sourceid>
<goals>
<goal>add-sourcegoal>
goals>
execution>
<execution>
<id>scala-compile-firstid>
<phase>process-resourcesphase>
<goals>
<goal>compilegoal>
goals>
execution>
<execution>
<id>scala-test-compile-firstid>
<phase>process-test-resourcesphase>
<goals>
<goal>testCompilegoal>
goals>
execution>
<execution>
<id>attach-scaladocsid>
<phase>verifyphase>
<goals>
<goal>doc-jargoal>
goals>
execution>
executions>
<configuration>
<scalaVersion>2.11.8scalaVersion>
<recompileMode>incrementalrecompileMode>
<useZincServer>trueuseZincServer>
<args>
<arg>-uncheckedarg>
<arg>-deprecationarg>
<arg>-featurearg>
args>
<javacArgs>
<javacArg>-sourcejavacArg>
<javacArg>${java.version}javacArg>
<javacArg>-targetjavacArg>
<javacArg>${java.version}javacArg>
<javacArg>-Xlint:all,-serial,-pathjavacArg>
javacArgs>
configuration>
plugin>解决将依赖一起打包进目标Jar包
使用插件maven-shade-plugin
<plugin>
<groupId>org.apache.maven.pluginsgroupId>
<artifactId>maven-shade-pluginartifactId>
<version>1.4version>
<configuration>
<createDependencyReducedPom>truecreateDependencyReducedPom>
configuration>
<executions>
<execution>
<phase>packagephase>
<goals>
<goal>shadegoal>
goals>
<configuration>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>mainClass>
transformer>
transformers>
configuration>
execution>
executions>
plugin>
那么一份完整的pom文件的结构应该是这样的：
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0modelVersion>
<groupId>com.neu.edugroupId>
<artifactId>wordcount1artifactId>
<version>1.0-SNAPSHOTversion>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.pluginsgroupId>
<artifactId>maven-shade-pluginartifactId>
<version>1.4version>
<configuration>
<createDependencyReducedPom>truecreateDependencyReducedPom>
configuration>
<executions>
<execution>
<phase>packagephase>
<goals>
<goal>shadegoal>
goals>
<configuration>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>mainClass>
transformer>
transformers>
configuration>
execution>
executions>
plugin>
<plugin>
<groupId>net.alchim31.mavengroupId>
<artifactId>scala-maven-pluginartifactId>
<version>3.2.2version>
<executions>
<execution>
<id>eclipse-add-sourceid>
<goals>
<goal>add-sourcegoal>
goals>
execution>
<execution>
<id>scala-compile-firstid>
<phase>process-resourcesphase>
<goals>
<goal>compilegoal>
goals>
execution>
<execution>
<id>scala-test-compile-firstid>
<phase>process-test-resourcesphase>
<goals>
<goal>testCompilegoal>
goals>
execution>
<execution>
<id>attach-scaladocsid>
<phase>verifyphase>
<goals>
<goal>doc-jargoal>
goals>
execution>
executions>
<configuration>
<scalaVersion>2.11.8scalaVersion>
<recompileMode>incrementalrecompileMode>
<useZincServer>trueuseZincServer>
<args>
<arg>-uncheckedarg>
<arg>-deprecationarg>
<arg>-featurearg>
args>
<javacArgs>
<javacArg>-sourcejavacArg>
<javacArg>${java.version}javacArg>
<javacArg>-targetjavacArg>
<javacArg>${java.version}javacArg>
<javacArg>-Xlint:all,-serial,-pathjavacArg>
javacArgs>
configuration>
plugin>
plugins>
build>
<dependencies>
dependencies>
project>