Spark-Shell起動時に多くのエラーが発生する

Question

"brew install apache-spark"を使用してsparkをダウンロードしました。スパークシェルを始めると、私は大量のエラーが発生します。私は火花セッション作成しようとすると：

val spark = SparkSession.builder().appName("Spark Postgresql Example").getOrCreate() 

を私は次のエラーを取得：

Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------ 
java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@73a116d, see the next exception for details. 

Caused by: ERROR XJ040: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@73a116d, see the next exception for details. 

org.datanucleus.exceptions.NucleusDataStoreException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------ 
java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@73a116d, see the next exception for details. 

Caused by: java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------ 
java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@73a116d, see the next exception for details. 

Nested Throwables StackTrace: 
java.sql.SQLException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------ 
java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@73a116d, see the next exception for details. 

17/07/18 13:12:35 WARN HiveMetaStore: Retrying creating default database after error: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------ 
java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@73a116d, see the next exception for details. 

javax.jdo.JDOFatalDataStoreException: Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------ 
java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@73a116d, see the next exception for details. 

17/07/18 13:12:35 ERROR Schema: Failed initialising database. 
Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=metastore_db;create=true, username = APP. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------ 
java.sql.SQLException: Failed to start database 'metastore_db' with class loader org.apache.spark.sql.hive.client.IsolatedClientLoader$$anon$1@73a116d, see the next exception for details. 

とかなり詳細..

scala> import spark.implicits._ 
<console>:18: error: not found: value spark 
    import spark.implicits._ 
     ^

Answer 1

このエラーは、スパーク・シェルがない場合に発生するように見えます新しいセッションがspark-shellを起動します。spark-shellを再起動します。

まだ起動していない場合あなたがセッション

var sparkSession = org.apache.spark.sql.SparkSessionbuilder.getOrCreate 
var sparkContext = sparkSession.sparkContext 

を作成するには、これを試すことができます起こっあなたがmetastore_dbを削除してみてくださいすることができます/ dbex.lckこれはあなたの問題

を修正するあなたはまた、{SPARK_HOME}/confの中でハイブ-site.xmlのを設定することができ、現在のディレクトリにmetastore_dbという名前のメタストアとwarehouseというフォルダが自動的に作成されます。spark-shellを起動しているディレクトリで権限の問題を修正してください。