1
kafka-clients-0.8.2.1.jarを使用していて、クラスタがまったく稼働していない場合、送信がブロックされることがわかりました。私はより関連性の高い情報をオンラインで見つけましたKafka 0.8.2 new producer blocking on metadataと多くの人々が直面している既知の問題だとわかりました。 しかし、いくつかのオプションを試してみましたが、助けてくれませんでした。 1. KafkaProducer.partitionsFor（）を呼び出してもブロックされます。 2. producer.typeをasyncに設定しましたが、認識されません。クラスタ停止時のKafkaPublisherブロック
私が達成したいのは、クラスタがダウンしていることを検出し、最大限のメッセージをキューに入れて、クラスタが戻ったときに送信することです。複雑すぎる場合は、少なくともアプリケーションがすべてのメッセージをキューに入れてメモリを使い果たしてしまうため、ブロックされません。
コード：
System.out.println("props:"
producer = new KafkaProducer<String, String>(props);
producer.partitionsFor(record.topic());
producer.send(record, new Callback() {
ログイン：内部バッファがいっぱいになったとき
props:{queue.size=1000, reconnect.backoff.ms=10000, request.timeout.ms=1000, bootstrap.servers=tstaapp001:59092,ewdlxsrv283:59092,devcapp001:59092, value.serializer=org.apache.kafka.common.serialization.StringSerializer, request.required.acks=1, buffer.memory=33554432, retries=0, producer.type=async, key.serializer=org.apache.kafka.common.serialization.StringSerializer, linger.ms=1, topic.metadata.refresh.interval.ms=1000, batch.size=16384, timeout.ms=10000}
20:44:33.584 [RTEMPool-1-RTEMThread-4] DEBUG o.a.k.c.producer.internals.Metadata - Updated cluster metadata version 1 to Cluster(nodes = [Node(tstaapp001, 59092), Node(devcapp001, 59092), Node(ewdlxsrv283, 59092)], partitions = [])
20:44:33.603 [kafka-producer-network-thread | producer-1] DEBUG o.a.k.c.producer.internals.Sender - Starting Kafka producer I/O thread.
20:44:33.603 [RTEMPool-1-RTEMThread-4] WARN o.a.k.c.producer.ProducerConfig - The configuration topic.metadata.refresh.interval.ms = null was supplied but isn't a known config.
20:44:33.603 [RTEMPool-1-RTEMThread-4] WARN o.a.k.c.producer.ProducerConfig - The configuration request.timeout.ms = null was supplied but isn't a known config.
20:44:33.603 [RTEMPool-1-RTEMThread-4] WARN o.a.k.c.producer.ProducerConfig - The configuration producer.type = null was supplied but isn't a known config.
20:44:33.604 [RTEMPool-1-RTEMThread-4] WARN o.a.k.c.producer.ProducerConfig - The configuration request.required.acks = null was supplied but isn't a known config.
20:44:33.604 [RTEMPool-1-RTEMThread-4] WARN o.a.k.c.producer.ProducerConfig - The configuration queue.size = null was supplied but isn't a known config.
20:44:33.604 [RTEMPool-1-RTEMThread-4] DEBUG o.a.k.clients.producer.KafkaProducer - Kafka producer started
20:44:42.958 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:42.959 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Init connection to node -2 for sending metadata request in the next iteration
20:44:42.959 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node -2 at ewdlxsrv283:59092.
20:44:42.964 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:43.063 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:43.163 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:43.264 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:43.364 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:43.464 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:43.564 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:43.664 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:43.764 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:43.864 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:43.964 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:44.064 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:44.164 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:44.264 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:44.364 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:44.464 [kafka-producer-network-thread | producer-1] DEBUG o.apache.kafka.clients.NetworkClient - Trying to send metadata request to node -2
20:44:44.479 [kafka-producer-network-thread | producer-1] WARN o.a.kafka.common.network.Selector - Error in I/O with null
java.net.ConnectException: Connection refused: no further information