1
私はHortonworks tutorial.Butに基づいて新しいSparkアプリケーションを作成しようとしていますが、build.sbtは必要なモジュールをインポートしようとすると以下のエラーを投げかけます。スパーク:sbt依存性エラー
私はSCALA - 2.11.4とSBT 0.13をインストールしました。
Error:Error while importing SBT project:<br/>...<br/><pre>[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Unresolved dependencies path:
[warn] org.scala-lang:scala-library:2.11.4 ((sbt.Classpaths) Defaults.scala#L1340)
[warn] +- sparktutorialscala:sparktutorialscala_2.11:1.0
[warn] org.apache.spark:spark-core_2.11:2.1.0 (C:\Users\USERNAME\IdeaProjects\HelloWorld\build.sbt#L7-13)
[warn] +- sparktutorialscala:sparktutorialscala_2.11:1.0
[warn] org.scala-lang:scala-compiler:2.11.4
[warn] +- sparktutorialscala:sparktutorialscala_2.11:1.0
[trace] Stack trace suppressed: run 'last *:ssExtractDependencies' for the full output.
[trace] Stack trace suppressed: run 'last *:update' for the full output.
[error] (*:ssExtractDependencies) sbt.ResolveException: unresolved dependency: org.scala-lang#scala-library;2.11.4: not found
[error] unresolved dependency: org.apache.spark#spark-core_2.11;2.1.0: not found
[error] unresolved dependency: org.scala-lang#scala-compiler;2.11.4: not found
[error] (*:update) sbt.ResolveException: unresolved dependency: org.scala-lang#scala-library;2.11.4: not found
[error] unresolved dependency: org.apache.spark#spark-core_2.11;2.1.0: not found
[error] unresolved dependency: org.scala-lang#scala-compiler;2.11.4: not found
[error] Total time: 66 s, completed 17-Oct-2017 10:49:23
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=384M; support was removed in 8.0
Picked up _JAVA_OPTIONS: -Djava.net.preferIPv4Stack=true</pre><br/>See complete
build.sbt
name := "sparkTutorialScala"
version := "1.0"
scalaVersion := "2.11.4"
libraryDependencies ++= {
val sparkVer = "2.1.0"
Seq(
"org.apache.spark" %% "spark-core" % sparkVer % "provided" withSources()
)
}
更新すると、エラーメッセージ:
[error] unresolved dependency: org.apache.spark#spark-core_2.11;2.2.0: public: unable to get resource for org/apache/spark#spark-core_2.11;2.2.0: res=https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.11/2.2.0/spark-core_2.11-2.2.0.pom: java.net.ConnectException: Failed to connect to repo1.maven.org/151.101.60.209:443
[error] unresolved dependency: org.scala-lang#scala-compiler;2.11.8: public: unable to get resource for org/scala-lang#scala-compiler;2.11.8: res=https://repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.11.8/scala-compiler-2.11.8.pom: java.net.ConnectException: Failed to connect to repo1.maven.org/151.101.60.209:443
[error] (*:ssExtractDependencies) sbt.librarymanagement.ResolveException: unresolved dependency: org.scala-lang#scala-library;2.11.8: public: unable to get resource for org/scala-lang#scala-library;2.11.8: res=https://repo1.maven.org/maven2/org/scala-lang/scala-library/2.11.8/scala-library-2.11.8.pom: java.net.ConnectException: Failed to connect to repo1.maven.org/151.101.60.209:443
[error] unresolved dependency: org.apache.spark#spark-core_2.11;2.2.0: public: unable to get resource for org/apache/spark#spark-core_2.11;2.2.0: res=https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.11/2.2.0/spark-core_2.11-2.2.0.pom: java.net.ConnectException: Failed to connect to repo1.maven.org/151.101.60.209:443
[error] unresolved dependency: org.scala-lang#scala-compiler;2.11.8: public: unable to get resource for org/scala-lang#scala-compiler;2.11.8: res=https://repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.11.8/scala-compiler-2.11.8.pom: java.net.ConnectException: Failed to connect to repo1.maven.org/151.101.60.209:443
[error] Total time: 9 s, completed 19-Oct-2017 15:11:47
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=384M; support was removed in 8.0
Picked up _JAVA_OPTIONS: -Djava.net.preferIPv4Stack=true</pre><br/>See complete log in <a href="file:/C:/Users/
あなたのsbtファイルを表示してください –
@yehor私の編集した投稿をSBTにチェックしてください – theDbGuy
あなたのプロジェクトにあなたが持っているすべてを表示していますか?エラー: '未解決の依存関係:org.apache.spark#spark-core_2.11_2.11; 2.1.0:見つからない' _2.11'が2回繰り返される(!) –