1
Phoenixで 'TEST'と呼ばれるテーブルを作成しました。このテーブルをPhoenixからクエリでき、HBaseのシェルでスキャンできます。私は次のようにフェニックス・スパークライブラリを使用しようとしているが、データフレームが満たされていない：Phoenix/SparkでDataFrameを読み込めません
import org.apache.hadoop.conf.Configuration
import org.apache.hadoop.hbase.HBaseConfiguration
import org.apache.phoenix.spark._
val hadoopConf: Configuration = new Configuration()
val hbConf: Configuration = HBaseConfiguration.create(hadoopConf)
val df = sqlContext.phoenixTableAsDataFrame("TEST", Array("foo", "bar"), conf = hbConf)
代わりに、私は何を取得することは以下の通りです：
16/05/11 11:10:47 INFO MemoryStore: ensureFreeSpace(413840) called with curMem=0, maxMem=4445479895
16/05/11 11:10:47 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 404.1 KB, free 4.1 GB)
16/05/11 11:10:47 INFO MemoryStore: ensureFreeSpace(27817) called with curMem=413840, maxMem=4445479895
16/05/11 11:10:47 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 27.2 KB, free 4.1 GB)
16/05/11 11:10:47 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on localhost:39319 (size: 27.2 KB, free: 4.1 GB)
16/05/11 11:10:47 INFO SparkContext: Created broadcast 0 from newAPIHadoopRDD at PhoenixRDD.scala:41
16/05/11 11:10:47 INFO RecoverableZooKeeper: Process identifier=hconnection-0x72187492 connecting to ZooKeeper ensemble=localhost:2181
16/05/11 11:10:47 INFO ZooKeeper: Client environment:zookeeper.version=3.4.6-2950--1, built on 09/30/2015 17:44 GMT
16/05/11 11:10:47 INFO ZooKeeper: Client environment:host.name=some.server.com
16/05/11 11:10:47 INFO ZooKeeper: Client environment:java.version=1.8.0_40
16/05/11 11:10:47 INFO ZooKeeper: Client environment:java.vendor=Oracle Corporation
16/05/11 11:10:47 INFO ZooKeeper: Client environment:java.home=/usr/jdk64/jdk1.8.0_40/jre
16/05/11 11:10:47 INFO ZooKeeper: Client environment:java.class.path=/usr/hdp/2.3.2.0-2950/spark/conf/:/usr/hdp/2.3.2.0-2950/spark/lib/spark-assembly-1.4.1.2.3.2.0-2950-hadoop2.7.1.2.3.2.0-2950.jar:/usr/hdp/2.3.2.0-2950/spark/lib/datanucleus-api-jdo-3.2.6.jar:/usr/hdp/2.3.2.0-2950/spark/lib/datanucleus-core-3.2.10.jar:/usr/hdp/2.3.2.0-2950/spark/lib/datanucleus-rdbms-3.2.9.jar:/usr/hdp/current/hadoop-client/conf/:/usr/hdp/current/hadoop-client/hadoop-azure.jar:/usr/hdp/current/hadoop-client/lib/azure-storage-2.2.0.jar
16/05/11 11:10:47 INFO ZooKeeper: Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
16/05/11 11:10:47 INFO ZooKeeper: Client environment:java.io.tmpdir=/tmp
16/05/11 11:10:47 INFO ZooKeeper: Client environment:java.compiler=<NA>
16/05/11 11:10:47 INFO ZooKeeper: Client environment:os.name=Linux
16/05/11 11:10:47 INFO ZooKeeper: Client environment:os.arch=amd64
16/05/11 11:10:47 INFO ZooKeeper: Client environment:os.version=3.10.0-327.10.1.el7.x86_64
16/05/11 11:10:47 INFO ZooKeeper: Client environment:user.name=dude
16/05/11 11:10:47 INFO ZooKeeper: Client environment:user.home=/home/dude
16/05/11 11:10:47 INFO ZooKeeper: Client environment:user.dir=/home/dude
16/05/11 11:10:47 INFO ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=90000 watcher=hconnection-0x721874920x0, quorum=localhost:2181, baseZNode=/hbase
16/05/11 11:10:47 INFO ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
16/05/11 11:10:47 INFO ClientCnxn: Socket connection established to localhost/127.0.0.1:2181, initiating session
16/05/11 11:10:47 INFO ClientCnxn: Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x25494c0cb650086, negotiated timeout = 40000
16/05/11 11:10:47 INFO Metrics: Initializing metrics system: phoenix
16/05/11 11:10:47 INFO MetricsConfig: loaded properties from hadoop-metrics2.properties
16/05/11 11:10:47 INFO MetricsSystemImpl: Scheduled snapshot period at 60 second(s).
16/05/11 11:10:47 INFO MetricsSystemImpl: phoenix metrics system started
16/05/11 11:10:48 INFO RecoverableZooKeeper: Process identifier=hconnection-0xd2eddc2 connecting to ZooKeeper ensemble=localhost:2181
16/05/11 11:10:48 INFO ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=90000 watcher=hconnection-0xd2eddc20x0, quorum=localhost:2181, baseZNode=/hbase
16/05/11 11:10:48 INFO ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
16/05/11 11:10:48 INFO ClientCnxn: Socket connection established to localhost/127.0.0.1:2181, initiating session
16/05/11 11:10:48 INFO ClientCnxn: Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x25494c0cb650087, negotiated timeout = 40000
16/05/11 11:11:36 INFO RpcRetryingCaller: Call exception, tries=10, retries=35, started=48168 ms ago, cancelled=false, msg=
16/05/11 11:11:56 INFO RpcRetryingCaller: Call exception, tries=11, retries=35, started=68312 ms ago, cancelled=false, msg=
16/05/11 11:12:16 INFO RpcRetryingCaller: Call exception, tries=12, retries=35, started=88338 ms ago, cancelled=false, msg=
16/05/11 11:12:36 INFO RpcRetryingCaller: Call exception, tries=13, retries=35, started=108450 ms ago, cancelled=false, msg=
16/05/11 11:12:56 INFO RpcRetryingCaller: Call exception, tries=14, retries=35, started=128530 ms ago, cancelled=false, msg=
16/05/11 11:13:16 INFO RpcRetryingCaller: Call exception, tries=15, retries=35, started=148547 ms ago, cancelled=false, msg=
16/05/11 11:13:37 INFO RpcRetryingCaller: Call exception, tries=16, retries=35, started=168741 ms ago, cancelled=false, msg=
16/05/11 11:13:57 INFO RpcRetryingCaller: Call exception, tries=17, retries=35, started=188856 ms ago, cancelled=false, msg=
私はthis postを見つけたが、私はすでに上記の方法を使用しており、HBase設定をパスしています。私は間違って何をしていますか？
興味深いことに、私のZKクォーラムサーバーはローカルホストではなく、INFOメッセージにローカルホストとして表示されるように見えますが、2台のサーバーの一覧です。私はそれが表示されるはずのものかどうかはわかりません。 hbase.zookeeper.quorumパラメータはhbase-site.xmlに正しく設定されており、hbConfをチェックするとリストに表示されます。また、メッセージに/hbaseが表示されていても、zookeeper.znode.parentは/hbase-unsecureに設定されています。 Phoenix-Sparkは単にそれらを無視しますか？
私はHBase APIを直接使用できますが、DataFrameとしてすぐにデータを読み込むことができるので、Phoenixを使用するといいでしょう。