Scala编程环境搭建以及Maven编译工具
2017-07-26 本文已影响0人
小甜瓜Melon
在mac下使用maven对Scala代码进行打包。
mac编程环境：
- 安装Scala （官网下载安装，tarball安装）；---->这个安装完成之后便可以使用Scala交互编程；
- 安装Java（Oracle官网下载，一路回车）；--->Scala编程基础。
- Scala编码环境Intellij IDEA（官网下载社区版本，一路回车），安装完成后IDEA自动检测到interpreter为Java(java version "1.8.0_141")，创建Scala工程和创建Scala File，发现no scala sdk in module，点击IDEA编辑栏的检测安装，将会自动下载完成之后，将不再报错。
- Intellij IDEA常用设置
- 设置View中的Tool Bar和Tool Button，方便调试。
- 设置字体Edit-->Font
- 设置外观 Appearance;
maven工具的安装
- 安装brew(mac专用，类似于centos中的yum)，终端执行即可成功安装。
$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
- 安装 wget，执行
brew install wget即可成功安装。 - 安装maven，执行
brew install maven即可成功安装。执行mvn --help可看到帮助。
创建Scala工程
此处略去。
scala工程中包含有src包，src包包括了代码。
maven工具的使用
maven工具可以打包Java代码和Scala代码，完成后会生成一个.jar文件，此文件即可用来提交执行。里面主要包含了jar包的启动入口和运行环境，以及依赖包。
在mac终端执行maven clean package即可打包成功。
打包成功后会出现一个可执行的文件.jar。
若出现错误，则会报错。
报错示例：The specified file '~/.m2/ik-analyzer-3.2.8.jar' not exists
MacBook-Pro:~/.m2$ mvn install:install-file -Dfile=ik-analyzer-3.2.8.jar -DgroupId=org.wltea.ik-analyzer -DartifactId=ik-analyzer -Dversion=3.2.8 -Dpackaging=jar
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-install-plugin:2.4:install-file (default-cli) @ standalone-pom ---
[ERROR] The specified file '~/.m2/ik-analyzer-3.2.8.jar' not exists
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.446 s
[INFO] Finished at: 2017-07-26T18:52:38+08:00
[INFO] Final Memory: 6M/155M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:2.4:install-file (default-cli) on project standalone-pom: The specified file '~/.m2/ik-analyzer-3.2.8.jar' not exists -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
一般的报错都是出现了依赖包的缺失，对照报错信息。查找相应的依赖包，放到对应的目录下面。这里报错信息有两个来源：一是shell终端的报错信息。二是在Intellij IDEA中查看缺失了哪些依赖jar包。
第一次报错之后出现了如下以下三个包的缺失，安装之。
MacBook-Pro:~/Desktop$ mvn install:install-file -Dfile=graphframes-0.2.0.jar -DgroupId=org.graphframes -DartifactId=graphframes -Dversion=0.2.0 -Dpackaging=jar
MacBook-Pro:~/Desktop$ mvn install:install-file -Dfile=ik-analyzer-3.2.8.jar -DgroupId=org.wltea.ik-analyzer -DartifactId=ik-analyzer -Dversion=3.2.8 -Dpackaging=jar
MacBook-Pro:~/Desktop$ mvn install:install-file -Dfile=reactivemongo_2.10-0.11.7.jar -DgroupId=org.reactivemongo -DartifactId=reactivemongo_2.10 -Dversion=0.11.7 -Dpackaging=jar
说明：Intellij IDEA视图，标注红线的即为缺失依赖包。
image.png
（小插曲：markdown改变文件大小调整后面的数字即可。例如本图形，原来为1280，现在调整为500，大小比较合适）
依赖包的位置全部包含在：~/.m2/repository/org/
安装成功后再次打包即可
MacBook-Pro:~/Desktop/catalog2/catalog3/catalog4/$ mvn clean package
这里需要特别注意：这句命令必须在文件所在的目录下执行，此处为catalog4，即项目所在的目录中最顶层，最顶层包含了pom.xml文件。
Project Object Model，项目对象模型。通过xml格式保存的pom.xml文件。作用类似ant的build.xml文件，功能更强大。该文件用于管理：源代码、配置文件、开发者的信息和角色、问题追踪系统、组织信息、项目授权、项目的url、项目的依赖关系等等。
一个完整的pom.xml文件，放置在项目的根目录下。
pom.xml详解
执行完成后生成的文件位于target目录下中
MacBook-Pro:~/Desktop/catalog2/catalog3/catalog4/target$ ll
-rw-r--r-- 1 user group 2107973 7 26 18:58 data-mgr-1.0.0-SNAPSHOT.jar
.jar文件即为可执行文件。
spark-submit此文件即可运行任务。
（正文完）
附件：打包过程。
MacBook-Pro:~/Desktop/catalog2/catalog3/catalog4/$ mvn clean package
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.welab.data:data-mgr:jar:1.0.0-SNAPSHOT
[WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-jar-plugin is missing. @ line 336, column 21
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building data-mgr 1.0.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[WARNING] The POM for com.typesafe.play:play-iteratees_2.10:jar:2.3.5 is missing, no dependency information available
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ data-mgr ---
[INFO]
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforce-versions) @ data-mgr ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ data-mgr ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 6 resources
[INFO]
[INFO] --- scala-maven-plugin:3.1.6:compile (scala-compile-first) @ data-mgr ---
[WARNING] Missing POM for com.typesafe.play:play-iteratees_2.10:jar:2.3.5
Downloading: https://repo.maven.apache.org/maven2/org/eclipse/jetty/aggregate/jetty-all/7.6.0.v20120127/jetty-all-7.6.0.v20120127.pom
Downloaded: https://repo.maven.apache.org/maven2/org/eclipse/jetty/aggregate/jetty-all/7.6.0.v20120127/jetty-all-7.6.0.v20120127.pom (8.4 kB at 5.1 kB/s)
Downloading: https://repo.maven.apache.org/maven2/org/eclipse/jetty/aggregate/jetty-aggregate-project/7.6.0.v20120127/jetty-aggregate-project-7.6.0.v20120127.pom
Downloaded: https://repo.maven.apache.org/maven2/org/eclipse/jetty/aggregate/jetty-aggregate-project/7.6.0.v20120127/jetty-aggregate-project-7.6.0.v20120127.pom (1.5 kB at 2.5 kB/s)
Downloading: https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-project/7.6.0.v20120127/jetty-project-7.6.0.v20120127.pom
Downloaded: https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-project/7.6.0.v20120127/jetty-project-7.6.0.v20120127.pom (22 kB at 35 kB/s)
Downloading: https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-parent/19/jetty-parent-19.pom
Downloaded: https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-parent/19/jetty-parent-19.pom (20 kB at 33 kB/s)
Downloading: https://repo.maven.apache.org/maven2/org/apache/geronimo/specs/geronimo-jta_1.1_spec/1.1.1/geronimo-jta_1.1_spec-1.1.1.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/geronimo/specs/geronimo-jta_1.1_spec/1.1.1/geronimo-jta_1.1_spec-1.1.1.pom (2.1 kB at 3.5 kB/s)
Downloading: https://repo.maven.apache.org/maven2/org/apache/geronimo/specs/specs/1.4/specs-1.4.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/geronimo/specs/specs/1.4/specs-1.4.pom (12 kB at 20 kB/s)
Downloading: https://repo.maven.apache.org/maven2/org/apache/geronimo/genesis/config/project-config/1.2/project-config-1.2.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/geronimo/genesis/config/project-config/1.2/project-config-1.2.pom (22 kB at 45 kB/s)
Downloading: https://repo.maven.apache.org/maven2/org/apache/geronimo/genesis/config/config/1.2/config-1.2.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/geronimo/genesis/config/config/1.2/config-1.2.pom (1.8 kB at 2.8 kB/s)
Downloading: https://repo.maven.apache.org/maven2/org/apache/geronimo/genesis/genesis/1.2/genesis-1.2.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/geronimo/genesis/genesis/1.2/genesis-1.2.pom (11 kB at 18 kB/s)
Downloading: https://repo.maven.apache.org/maven2/javax/mail/mail/1.4.1/mail-1.4.1.pom
Downloaded: https://repo.maven.apache.org/maven2/javax/mail/mail/1.4.1/mail-1.4.1.pom (995 B at 1.7 kB/s)
Downloading: https://repo.maven.apache.org/maven2/org/apache/geronimo/specs/geronimo-jaspic_1.0_spec/1.0/geronimo-jaspic_1.0_spec-1.0.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/geronimo/specs/geronimo-jaspic_1.0_spec/1.0/geronimo-jaspic_1.0_spec-1.0.pom (2.9 kB at 4.7 kB/s)
Downloading: https://repo.maven.apache.org/maven2/org/apache/geronimo/genesis/genesis-java1.4-flava/2.0/genesis-java1.4-flava-2.0.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/geronimo/genesis/genesis-java1.4-flava/2.0/genesis-java1.4-flava-2.0.pom (5.5 kB at 9.0 kB/s)
Downloading: https://repo.maven.apache.org/maven2/org/apache/geronimo/genesis/genesis-default-flava/2.0/genesis-default-flava-2.0.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/geronimo/genesis/genesis-default-flava/2.0/genesis-default-flava-2.0.pom (18 kB at 29 kB/s)
Downloading: https://repo.maven.apache.org/maven2/org/apache/geronimo/genesis/genesis/2.0/genesis-2.0.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/geronimo/genesis/genesis/2.0/genesis-2.0.pom (18 kB at 32 kB/s)
Downloading: https://repo.maven.apache.org/maven2/org/apache/geronimo/specs/geronimo-annotation_1.0_spec/1.1.1/geronimo-annotation_1.0_spec-1.1.1.pom
Downloaded: https://repo.maven.apache.org/maven2/org/apache/geronimo/specs/geronimo-annotation_1.0_spec/1.1.1/geronimo-annotation_1.0_spec-1.1.1.pom (2.2 kB at 3.4 kB/s)
Downloading: https://repo.maven.apache.org/maven2/asm/asm-commons/3.1/asm-commons-3.1.pom
Downloaded: https://repo.maven.apache.org/maven2/asm/asm-commons/3.1/asm-commons-3.1.pom (436 B at 859 B/s)
Downloading: https://repo.maven.apache.org/maven2/asm/asm-tree/3.1/asm-tree-3.1.pom
Downloaded: https://repo.maven.apache.org/maven2/asm/asm-tree/3.1/asm-tree-3.1.pom (425 B at 705 B/s)
[WARNING] Expected all dependencies to require Scala version: 2.10.4
[WARNING] com.twitter:chill_2.10:0.5.0 requires scala version: 2.10.4
[WARNING] com.typesafe.akka:akka-remote_2.10:2.3.11 requires scala version: 2.10.4
[WARNING] com.typesafe.akka:akka-slf4j_2.10:2.3.11 requires scala version: 2.10.4
[WARNING] org.apache.spark:spark-core_2.10:1.6.0 requires scala version: 2.10.5
[WARNING] Multiple versions of scala libraries detected!
[WARNING] Zinc server is not available at port 3030 - reverting to normal incremental compile
[INFO] Using incremental compilation
[INFO] 'compiler-interface' not yet compiled for Scala 2.10.4. Compiling...
[INFO] Compilation completed in 11.413 s
---省略号---
[INFO]
[INFO] --- scala-maven-plugin:3.1.6:testCompile (scala-test-compile-first) @ data-mgr ---
[WARNING] Expected all dependencies to require Scala version: 2.10.4
[WARNING] com.twitter:chill_2.10:0.5.0 requires scala version: 2.10.4
[WARNING] com.typesafe.akka:akka-remote_2.10:2.3.11 requires scala version: 2.10.4
[WARNING] com.typesafe.akka:akka-slf4j_2.10:2.3.11 requires scala version: 2.10.4
[WARNING] org.apache.spark:spark-core_2.10:1.6.0 requires scala version: 2.10.5
[WARNING] Multiple versions of scala libraries detected!
[INFO] No sources to compile
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ data-mgr ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ data-mgr ---
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ data-mgr ---
^@[INFO] Building jar: /Desktop/catalog1/catalog2/catalog3/catalog4/target/data-mgr-1.0.0-SNAPSHOT.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:03 min
[INFO] Finished at: 2017-07-26T18:58:00+08:00
[INFO] Final Memory: 44M/943M
[INFO] ------------------------------------------------------------------------
参考链接：pom.xml详解
