1
以下に言及するJavaコードによって複数のパーティションを持つトピックを作成しました。 Javaコードを実行すると KafkaはNoSuchElementExceptionを返します
private void existOrCreateTopicsWithPartition(Properties prop) {
String kafkaServer = prop.getProperty(KafkaProducerKey.BOOTSTRAP_SERVERS_CONFIG.getKey());
int replicationFactor = kafkaServer.split(BROKER_SEPERATION_REGEX).length;
int totalPartionCount;
ZkClient zkClient = new ZkClient(kafkaServer, Integer.MAX_VALUE, Integer.MAX_VALUE,new SerializableSerializer());
ZkUtils zkutils = new ZkUtils(zkClient, new ZkConnection(kafkaServer), false);
String producerTopics = prop.getProperty(KafkaProducerKey.PRODUCER_TOPICS.getKey());
String topics[] = producerTopics.split(TOPICS_SEPERATOR_REGEX);
for(String topic : topics){
topic = topic.trim();
if(!AdminUtils.topicExists(zkutils, topic)){
totalPartionCount = Integer.valueOf(prop.getProperty(topic+PARTITION_KEY, "1"));
AdminUtils.createTopic(zkutils, topic, totalPartionCount, replicationFactor, new Properties());
}
}
}
が正常に私は結果ではなくエラーを与えるカフカコンソールを使用
[email protected]:~/Downloads/kafka_2.9.2-0.8.1$ bin/kafka-topics.sh --zookeeper 192.168.56.101:2181 --topic sms_cat_2 --describe
上のトピックの説明を確認しました。言及コマンド以下のログ:私は春の統合の助けを借りて作成された消費者を実行した場合
Error while executing topic command next on empty iterator java.util.NoSuchElementException: next on empty iterator
at scala.collection.Iterator$$anon$3.next(Iterator.scala:27)
at scala.collection.Iterator$$anon$3.next(Iterator.scala:25)
at scala.collection.IterableLike$class.head(IterableLike.scala:90)
at scala.collection.immutable.Map$EmptyMap$.head(Map.scala:88)
at kafka.admin.TopicCommand$$anonfun$describeTopic$1.apply(TopicCommand.scala:147)
at kafka.admin.TopicCommand$$anonfun$describeTopic$1.apply(TopicCommand.scala:137)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:60)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at kafka.admin.TopicCommand$.describeTopic(TopicCommand.scala:137)
at kafka.admin.TopicCommand$.main(TopicCommand.scala:58)
at kafka.admin.TopicCommand.main(TopicCommand.scala)
はそれが
WARN NetworkClient:582 - Error while fetching metadata with correlation id 1 : {sms_cat_1=LEADER_NOT_AVAILABLE, sms_cat_2=INVALID_TOPIC_EXCEPTION}
のようなUPDATEに見える無限ループに警告を与えます。
bin/kafka-topics.sh --zookeeper 192.168.56.101:2181 --describe
Topic:__consumer_offsets PartitionCount:50 ReplicationFactor:1 Configs:segment.bytes=104857600,cleanup.policy=compact,compression.type=uncompressed
Topic: __consumer_offsets Partition: 0 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 1 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 2 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 3 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 4 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 5 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 6 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 7 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 8 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 9 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 10 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 11 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 12 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 13 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 14 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 15 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 16 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 17 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 18 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 19 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 20 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 21 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 22 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 23 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 24 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 25 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 26 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 27 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 28 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 29 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 30 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 31 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 32 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 33 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 34 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 35 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 36 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 37 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 38 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 39 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 40 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 41 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 42 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 43 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 44 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 45 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 46 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 47 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 48 Leader: 0 Replicas: 0 Isr: 0
Topic: __consumer_offsets Partition: 49 Leader: 0 Replicas: 0 Isr: 0 Error while executing topic command next on empty iterator java.util.NoSuchElementException: next on empty iterator
at scala.collection.Iterator$$anon$3.next(Iterator.scala:27)
at scala.collection.Iterator$$anon$3.next(Iterator.scala:25)
at scala.collection.IterableLike$class.head(IterableLike.scala:90)
at scala.collection.immutable.Map$EmptyMap$.head(Map.scala:88)
at kafka.admin.TopicCommand$$anonfun$describeTopic$1.apply(TopicCommand.scala:147)
at kafka.admin.TopicCommand$$anonfun$describeTopic$1.apply(TopicCommand.scala:137)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:60)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
at kafka.admin.TopicCommand$.describeTopic(TopicCommand.scala:137)
at kafka.admin.TopicCommand$.main(TopicCommand.scala:58)
at kafka.admin.TopicCommand.main(TopicCommand.scala)
提案はありますか?
おかげ
コマンド 'bin/kafka-topics.sh --zookeeper 192.168.56.101:2181 --describe'の結果を投稿できますか? – avr
更新された質問を見つける –
それは次のKafka Issue https://issues.apache.org/jira/browse/KAFKA-1737に近いと思われる – avr