java.lang.NoClassDefFoundError: org/apache/spark/SparkConf

[tyx11111]

package com.oreilly.learningsparkexamples.java;

import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaRDD;
import org.apache.spark.api.java.JavaSparkContext;

public class JavaSparkTest {
public static void main(String[] args) {
SparkConf conf = new SparkConf().setMaster("local").setAppName("My App");
JavaSparkContext sc = new JavaSparkContext(conf);
// JavaRDD rdd = sc.textFile("D:\工作\20171227\newdesc.xml");
// JavaRDD lineee = rdd.filter(line -> line.contains("desc"));
// System.out.println(lineee.count());
}
}

直接git到本地，然后依赖包也全部下载完毕了，但是始终报错找不到jar包：
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/SparkConf
at com.oreilly.learningsparkexamples.java.JavaSparkTest.main(JavaSparkTest.java:12)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.SparkConf
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 6 more

工具是使用的IntelliJ Idea

[qihouying]

遇到同样的问题

[robin-lai]

遇到同样的问题

[summer1897]

<dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-core_2.10</artifactId> <version>${spark.version}</version> <!--<scope>provided</scope>--> </dependency>
注释掉scope或将provided改为compile，因为最后打包提交的时候要用到这个jar包。

[olashile]

java.lang.ClassNotFoundException: org.apache.spark.SparkConf
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:436)

Still having this issue, has it been solved.

Below is my sbt
`name := "ProjectA"

version := "0.1"

//scalaVersion := "2.13.1"

scalaVersion := "2.11.8"

// https://mvnrepository.com/artifact/org.apache.spark/spark-core
libraryDependencies += "org.apache.spark" %% "spark-core" % "2.4.1"`