leftJoinWithCassandraTable spark-cassandra-connectorを使用

Question

spark-cassandra-connectorを使用して、cassandraにleftJoinWithCassandraTableを実装する方法。私はscala 2.11/DSE5.0.3/Spark 1.6.2を使用しています。

RDDFunctionsクラスに2.0のメソッドがあると思います。感想を聞かせてください。前もって感謝します。

Answer 1

これをチェックしましたか？leftJoinWithCassandraTable？

/** 
    * Uses the data from [[org.apache.spark.rdd.RDD RDD]] to left join with a Cassandra table without 
    * retrieving the entire table. 
    * Any RDD which can be used to saveToCassandra can be used to leftJoinWithCassandra as well as any 
    * RDD which only specifies the partition Key of a Cassandra Table. This method executes single 
    * partition requests against the Cassandra Table and accepts the functional modifiers that a 
    * normal [[com.datastax.spark.connector.rdd.CassandraTableScanRDD]] takes. 
    * 
    * By default this method only uses the Partition Key for joining but any combination of columns 
    * which are acceptable to C* can be used in the join. Specify columns using joinColumns as a parameter 
    * or the on() method. 
    * 
    * Example With Prior Repartitioning: {{{ 
    * val source = sc.parallelize(keys).map(x => new KVRow(x)) 
    * val repart = source.repartitionByCassandraReplica(keyspace, tableName, 10) 
    * val someCass = repart.leftJoinWithCassandraTable(keyspace, tableName) 
    * }}} 
    * 
    * Example Joining on Clustering Columns: {{{ 
    * val source = sc.parallelize(keys).map(x => (x, x * 100)) 
    * val someCass = source.leftJoinWithCassandraTable(keyspace, wideTable).on(SomeColumns("key", "group")) 
    * }}} 
**/