Kafka consumer fetching metadata for topics failed -
i attempting write java client third party's kafka , zookeeper servers. able list , describe topics, when attempt read any, closedchannelexception
raised. reproduce them here command line client.
$ bin/kafka-console-consumer.sh --zookeeper 255.255.255.255:2181 --topic eventbustopic [2015-06-02 16:23:04,375] warn fetching topic metadata correlation id 0 topics [set(eventbustopic)] broker [id:1,host:some_host,port:9092] failed (kafka.client.clientutils$) java.nio.channels.closedchannelexception @ kafka.network.blockingchannel.send(blockingchannel.scala:100) @ kafka.producer.syncproducer.liftedtree1$1(syncproducer.scala:73) @ kafka.producer.syncproducer.kafka$producer$syncproducer$$dosend(syncproducer.scala:72) @ kafka.producer.syncproducer.send(syncproducer.scala:113) @ kafka.client.clientutils$.fetchtopicmetadata(clientutils.scala:58) @ kafka.client.clientutils$.fetchtopicmetadata(clientutils.scala:93) @ kafka.consumer.consumerfetchermanager$leaderfinderthread.dowork(consumerfetchermanager.scala:66) @ kafka.utils.shutdownablethread.run(shutdownablethread.scala:60) [2015-06-02 16:23:04,515] warn fetching topic metadata correlation id 0 topics [set(eventbustopic)] broker [id:0,host:some_host,port:9092] failed (kafka.client.clientutils$) java.nio.channels.closedchannelexception @ kafka.network.blockingchannel.send(blockingchannel.scala:100) @ kafka.producer.syncproducer.liftedtree1$1(syncproducer.scala:73) @ kafka.producer.syncproducer.kafka$producer$syncproducer$$dosend(syncproducer.scala:72) @ kafka.producer.syncproducer.send(syncproducer.scala:113) @ kafka.client.clientutils$.fetchtopicmetadata(clientutils.scala:58) @ kafka.client.clientutils$.fetchtopicmetadata(clientutils.scala:93) @ kafka.consumer.consumerfetchermanager$leaderfinderthread.dowork(consumerfetchermanager.scala:66) @ kafka.utils.shutdownablethread.run(shutdownablethread.scala:60)
alternate commands succeed:
$ bin/kafka-topics.sh --describe --zookeeper 255.255.255.255:2181 --topic eventbustopic topic:eventbustopic partitioncount:2 replicationfactor:1 configs: topic: eventbustopic partition: 0 leader: 1 replicas: 1 isr: 1 topic: eventbustopic partition: 1 leader: 0 replicas: 0 isr: 0 $ bin/kafka-topics.sh --list --zookeeper 255.255.255.255:2181 --topic eventbustopic eventbustopic
(the ips redacted , replaced 255.255.255.255)
when google exception, see issues on producer side -- indeed, source clientutils.fetchtopicmetadata
suggests used producers.
one concern have might product of network layout: packets mangled haproxy , sent on vpn.
what @ work here?
the broker tells client hostname should used produce/consume messages. default kafka uses hostname of system runs on. if hostname can not resolved client side exception.
you can try setting advertised.host.name
in kafka configuration hostname/address clients should use.
Comments
Post a Comment