Monday, March 6, 2017

Additional jar files when running Spark under Hadoop YARN mode (CDH 5.10.0 with Scala 2.10 and Spark 1.6.0)



lrwxrwxrwx 1 root root     91 Feb 23 15:02 spark-core_2.10-1.6.0-cdh5.10.0.jar -> /opt/cloudera/parcels/CDH-5.10.0-1.cdh5.10.0.p0.41/jars/spark-core_2.10-1.6.0-cdh5.10.0.jar
lrwxrwxrwx 1 root root     80 Feb 23 15:15 scala-library-2.10.6.jar -> /opt/cloudera/parcels/CDH-5.10.0-1.cdh5.10.0.p0.41/jars/scala-library-2.10.6.jar
lrwxrwxrwx 1 root root     37 Mar  6 14:06 commons-lang3-3.3.2.jar -> ../../../jars/commons-lang3-3.3.2.jar
-rw-r--r-- 1 root root 185676 Mar  6 14:11 typesafe-config-2.10.1.jar
lrwxrwxrwx 1 root root     55 Mar  6 14:26 akka-actor_2.10-2.2.3-shaded-protobuf.jar -> ../../../jars/akka-actor_2.10-2.2.3-shaded-protobuf.jar
lrwxrwxrwx 1 root root     56 Mar  6 14:28 akka-remote_2.10-2.2.3-shaded-protobuf.jar -> ../../../jars/akka-remote_2.10-2.2.3-shaded-protobuf.jar
lrwxrwxrwx 1 root root     55 Mar  6 14:29 akka-slf4j_2.10-2.2.3-shaded-protobuf.jar -> ../../../jars/akka-slf4j_2.10-2.2.3-shaded-protobuf.jar
lrwxrwxrwx 1 root root     70 Mar  6 14:40 spark-assembly-1.6.0-cdh5.10.0-hadoop2.6.0-cdh5.10.0.jar -> ../../../jars/spark-assembly-1.6.0-cdh5.10.0-hadoop2.6.0-cdh5.10.0.jar
[root@john2 lib]# pwd
/opt/cloudera/parcels/CDH-5.10.0-1.cdh5.10.0.p0.41/lib/hadoop-yarn/lib

No comments:

Post a Comment