0
Zookeeper & Kafkaの場合、16コアのVMに16ノードの& 16 GB RAMを搭載したVMに1つのノードをインストールしようとしています。 Zookeeperはf-syncの問題を表示しており、設定された6000msのタイムアウト時間の後にkafkaがタイムアウトを与えます。Zookeeperのf-syncの問題によりKafkaのタイムアウトが発生する
追加情報： 飼育係バージョン：3.4.6-1569965 カフカバージョン：エントリ以下2.11-0.9.0.1
はZKログにありました：
[2017-03-06 16:37:18,013] INFO Processed session termination for sessionid: 0x15aa58d60530000 (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-03-06 16:37:45,308] WARN fsync-ing the write ahead log in SyncThread:0 took 35773ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide (org.apache.zookeeper.server.persistence.FileTxnLog)
[2017-03-06 16:37:45,334] INFO Established session 0x15aa58d60530000 with negotiated timeout 6000 for client /127.0.0.1:44860 (org.apache.zookeeper.server.ZooKeeperServer)
[2017-03-06 16:37:45,340] WARN caught end of stream exception (org.apache.zookeeper.server.NIOServerCnxn)
EndOfStreamException: Unable to read additional data from client sessionid 0x15aa58d60530000, likely client has closed socket
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
at java.lang.Thread.run(Thread.java:745)
[2017-03-06 16:37:45,347] INFO Closed socket connection for client /127.0.0.1:44860 which had sessionid 0x15aa58d60530000 (org.apache.zookeeper.server.NIOServerCnxn)
[2017-03-06 16:37:50,360] WARN fsync-ing the write ahead log in SyncThread:0 took 5009ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide (org.apache.zookeeper.server.persistence.FileTxnLog)
カフカは、エラーをZKするタイムアウトを持っています：
[2017-03-06 16:10:11,115] FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 6000
at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1223)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:155)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:129)
at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:89)
at kafka.utils.ZkUtils$.apply(ZkUtils.scala:71)
at kafka.server.KafkaServer.initZk(KafkaServer.scala:278)
at kafka.server.KafkaServer.startup(KafkaServer.scala:168)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37)
at kafka.Kafka$.main(Kafka.scala:67)
at kafka.Kafka.main(Kafka.scala)
[2017-03-06 16:10:11,120] INFO shutting down (kafka.server.KafkaServer)
[2017-03-06 16:10:11,128] INFO shut down completed (kafka.server.KafkaServer)
[2017-03-06 16:10:11,129] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 6000
at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1223)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:155)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:129)
at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:89)
at kafka.utils.ZkUtils$.apply(ZkUtils.scala:71)
at kafka.server.KafkaServer.initZk(KafkaServer.scala:278)
at kafka.server.KafkaServer.startup(KafkaServer.scala:168)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:37)
at kafka.Kafka$.main(Kafka.scala:67)
at kafka.Kafka.main(Kafka.scala)
[2017-03-06 16:10:11,140] INFO shutting down (kafka.server.KafkaServer)
利用可能な回避策では既知の問題であれば、私がフォローアップできるリンク/フォーラムを親切に共有してください。
おかげ