kafka 的连接问题...

V2EX = way to explore

V2EX 是一个关于分享和探索的地方

现在注册

已注册用户请登录

这是一个创建于 2271 天前的主题，其中的信息可能已经有所发展或是发生改变。

目前是我要通过我的服务器连远程服务器的 kafka，我的服务器只作 producer，远程服务器把我的 ip 加入白名单了, 我用 telnet 也能通，可是用代码(python: ptkafka\kafka-python)就报这种错：

pykafka: 'Failed to connect newly created broker for ip:host'
kafka-python: 'NoBrokersAvailable'

目前是 broker 或者配置问题？另外我服务器不需要安装 kafka 了吧？

我才看了半天 kafka 菜鸡求教大佬们

第 1 条附言 · 2018-08-21 17:06:31 +08:00

后来加上api_version参数后就如下错误，找了好多说是kafka配置问题..

INFO:kafka.client:Bootstrap succeeded: found 1 brokers and 35 topics.
INFO:kafka.conn:<BrokerConnection node_id=bootstrap host=host:port <connected> [IPv4 ('host', port)]>: Closing connection.
DEBUG:kafka.conn:<BrokerConnection node_id=bootstrap host=host:port <disconnecting> [IPv4 ('host', port)]>: reconnect backoff 0.05978240071782098 after 1 failures
DEBUG:kafka.metrics.metrics:Added sensor with name bufferpool-wait-time
DEBUG:kafka.metrics.metrics:Added sensor with name batch-size
DEBUG:kafka.metrics.metrics:Added sensor with name compression-rate
DEBUG:kafka.metrics.metrics:Added sensor with name queue-time
DEBUG:kafka.metrics.metrics:Added sensor with name produce-throttle-time
DEBUG:kafka.metrics.metrics:Added sensor with name records-per-request
DEBUG:kafka.metrics.metrics:Added sensor with name bytes
DEBUG:kafka.metrics.metrics:Added sensor with name record-retries
DEBUG:kafka.metrics.metrics:Added sensor with name errors
DEBUG:kafka.metrics.metrics:Added sensor with name record-size-max
DEBUG:kafka.producer.sender:Starting Kafka producer I/O thread.
DEBUG:kafka.producer.kafka:Kafka producer started
DEBUG:kafka.producer.kafka:Requesting metadata update for topic enmonitor
DEBUG:kafka.client:Initializing connection to node 0 for metadata request
DEBUG:kafka.client:Initiating connection to node 0 at host_1:port
DEBUG:kafka.metrics.metrics:Added sensor with name node-0.bytes-sent
DEBUG:kafka.metrics.metrics:Added sensor with name node-0.bytes-received
DEBUG:kafka.metrics.metrics:Added sensor with name node-0.latency

第 2 条附言 · 2018-08-21 17:06:57 +08:00

附上一条附言
DEBUG:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <disconnected> [IPv4 None]>: creating new socket
DEBUG:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <disconnected> [IPv4 ('host_1', port)]>: setting socket option (6, 1, 1)
INFO:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <connecting> [IPv4 ('host_1', port)]>: connecting to host_1:port [('host_1', port) IPv4]
ERROR:kafka.conn:Connection attempt to <BrokerConnection node_id=0 host=host_1:port <connecting> [IPv4 ('host_1', port)]> timed out
INFO:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <connecting> [IPv4 ('host_1', port)]>: Closing connection. KafkaConnectionError: timeout
WARNING:kafka.client:Node 0 connection failed -- refreshing metadata
DEBUG:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <disconnecting> [IPv4 ('host_1', port)]>: reconnect backoff 0.04183829594241324 after 1 failures
DEBUG:kafka.client:Initializing connection to node 0 for metadata request
DEBUG:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <disconnected> [IPv4 ('host_1', port)]>: creating new socket
DEBUG:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <disconnected> [IPv4 ('host_1', port)]>: setting socket option (6, 1, 1)
INFO:kafka.conn:<BrokerConnection node_id=0 host=host_1:port <connecting> [IPv4 ('host_1', port)]>: connecting to host_1:port [('host_1', port) IPv4]
Traceback (most recent call last):
kafka.errors.KafkaTimeoutError: KafkaTimeoutError: Failed to update metadata after 60.0 secs.

Kafka

服务器

broker

菜鸡

3 条回复 • 2018-10-18 21:16:04 +08:00