jdbcを使用してSpark DataFrameをデータベースに書き込むことができません

Question

単純なデータフレームをOracleデータベースに書き込もうとしていますが、エラーメッセージが表示されます。私はケースクラスとリストを使用してデータフレームを構築します。私は、書き込み後にjdbcというメソッドを使用してOracleデータベースにデータを挿入できることを発見しました。 私はこのコードを試してみました：

case class MyClass(A: String, B: Int) 
val MyClass_List = List(MyClass("att1", 1), MyClass("att2", 2)) 

val MyClass_df = MyClass_List.toDF() 

MyClass_df.write 
      .mode("append") 
      .jdbc(url, tableTest, prop) 

を私は次のエラーを取得する：

17/07/12 14:57:04 ERROR JobScheduler: Error running job streaming job 1499864218000 ms.0 
java.lang.NullPointerException 
     at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:93) 
     at org.apache.spark.sql.execution.datasources.DataSource.write(DataSource.scala:426) 
     at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:215) 
     at org.apache.spark.sql.DataFrameWriter.jdbc(DataFrameWriter.scala:446) 
     at Test$$anonfun$1.apply(Test.scala:177) 
     at Test$$anonfun$1.apply(Test.scala:117) 
     at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:627) 
     at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:627) 
     at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51) 
     at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51) 
     at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51) 
     at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415) 
     at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50) 
     at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50) 
     at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50) 
     at scala.util.Try$.apply(Try.scala:192) 
     at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39) 
     at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254) 
     at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254) 
     at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254) 
     at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58) 
     at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253) 
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
     at java.lang.Thread.run(Thread.java:748) 
Exception in thread "main" java.lang.NullPointerException 
     at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:93) 
     at org.apache.spark.sql.execution.datasources.DataSource.write(DataSource.scala:426) 
     at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:215) 
     at org.apache.spark.sql.DataFrameWriter.jdbc(DataFrameWriter.scala:446) 
     at Test$$anonfun$1.apply(Test.scala:177) 
     at Test$$anonfun$1.apply(Test.scala:117) 
     at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:627) 
     at org.apache.spark.streaming.dstream.DStream$$anonfun$foreachRDD$1$$anonfun$apply$mcV$sp$3.apply(DStream.scala:627) 
     at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ForEachDStream.scala:51) 
     at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51) 
     at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(ForEachDStream.scala:51) 
     at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415) 
     at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply$mcV$sp(ForEachDStream.scala:50) 
     at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50) 
     at org.apache.spark.streaming.dstream.ForEachDStream$$anonfun$1.apply(ForEachDStream.scala:50) 
     at scala.util.Try$.apply(Try.scala:192) 
     at org.apache.spark.streaming.scheduler.Job.run(Job.scala:39) 
     at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply$mcV$sp(JobScheduler.scala:254) 
     at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254) 
     at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler$$anonfun$run$1.apply(JobScheduler.scala:254) 
     at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58) 
     at org.apache.spark.streaming.scheduler.JobScheduler$JobHandler.run(JobScheduler.scala:253) 
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
     at java.lang.Thread.run(Thread.java:748) 

私はスパークバージョン2.1.0を使用して、2つの列AとBとしての私のデータベースはそれぞれvarchar型と番号として入力。

アイデアはありますか？

Answer 1

実際、私はoracleのドライバにもかかわらずmysqlのドライバを使用していました。 私は、ドライバパッケージのいずれかが推奨されませんように「はoracle.jdbc.OracleDriver」である必要があります

prop.setProperty("driver", "oracle.jdbc.driver.OracleDriver") 

なく

prop.setProperty("driver", "com.mysql.jdbc.Driver") 

Answer 2

を使用する必要があります。

prop.setProperty("driver", "oracle.jdbc.OracleDriver")