我试图使用 statefulset 在 gke 的集群中部署 zookeeper 和 Kafka 服务器,但 Kafka 和 zookeeper 之间的通信失败,日志中出现错误消息。我想知道在 kubernetes 中设置 Kafka 的最简单方法是什么。

我尝试了以下配置,发现 Kafka 无法与 zookeeper 通信,但不知道为什么?我知道我可能需要 headless 服务,因为通信由 Kafka 和 zookeeper 自己处理。

动物园管理员

apiVersion: v1 
kind: Service 
metadata: 
  name: zookeeper 
spec: 
  type: LoadBalancer 
  selector: 
    app: zookeeper 
  ports: 
  - port: 2181 
    targetPort: client 
    name: zk-port 
  - port: 2888 
    targetPort: leader 
    name: zk-leader 
  - port: 3888 
    targetPort: election 
    name: zk-election 
--- 
apiVersion: apps/v1 
kind: StatefulSet 
metadata: 
  name: zookeeper 
spec: 
  replicas: 3 
  selector: 
    matchLabels: 
      app: zookeeper 
  serviceName: zookeeper 
  podManagementPolicy: Parallel 
  template: 
    metadata: 
      labels: 
        app: zookeeper 
    spec: 
      containers: 
        - name: zk-pod 
          image: zookeeper:latest 
          imagePullPolicy: Always 
          ports: 
            - name: client 
              containerPort: 2181 
            - name: leader 
              containerPort: 2888 
            - name: election 
              containerPort: 3888 
          env: 
            - name: ZOO_MY_ID 
              valueFrom: 
                fieldRef: 
                  fieldPath: metadata.name 
            - name: ZOO_TICK_TIME 
              value: "2000" 
            - name: ZOO_INIT_LIMIT 
              value: "5" 
            - name: ZOO_SYNC_LIMIT 
              value: "2" 
            - name: ZOO_SERVERS 
              value: zookeeper:2888:3888 

卡夫卡
apiVersion: apps/v1 
kind: StatefulSet 
metadata: 
  name: kafka 
spec: 
  replicas: 3 
  selector: 
    matchLabels: 
      app: kafka 
  serviceName: kafka-svc 
  podManagementPolicy: Parallel 
  template: 
    metadata: 
      labels: 
        app: kafka 
    spec: 
      containers: 
      - name: kafka 
        image: confluentinc/cp-kafka:latest 
        ports: 
          - containerPort: 9092 
            name: client 
        env: 
          - name: KAFKA_ZOOKEEPER_CONNECT 
            value: zookeeper:2181 
          - name: KAFKA_ADVERTISED_LISTENERS 
            value: kafka.default.svc.cluster.local:9092 
--- 
apiVersion: v1 
kind: Service 
metadata: 
  name: kafka-svc 
spec: 
  type: LoadBalancer 
  selector: 
    app: kafka 
  ports: 
  - port: 9092 
    targetPort: client 
    name: kfk-port 
--- 
apiVersion: policy/v1beta1 
kind: PodDisruptionBudget 
metadata: 
  name: kafka-pdb 
spec: 
  selector: 
    matchLabels: 
      app: kafka 
  minAvailable: 2 

我希望能够向某个主题发送消息并能够读回它们。我一直在使用 kafkacat测试连接。

请您参考如下方法:

这是 Official Kubernetes Documentation about StateFulsets 中指定的限制之一。 , 那

  • StatefulSets currently require a Headless Service to be responsible for the network identity of the Pods. You are responsible for creating this Service.


所以,正如你提到的,你需要 Headless Service并且您可以轻松地将 headless 服务 yaml 添加到您的配置顶部,类似于下面的两个 StatefulSet:
apiVersion: v1 
kind: Service 
metadata: 
  name: zookeeper 
  labels: 
    app: zookeeper 
spec: 
  ports: 
  - port: 2181  
    name: someport 
  clusterIP: None 
  selector: 
    app: zookeeper 

希望能帮助到你!


评论关闭
IT序号网

微信公众号号:IT虾米 (左侧二维码扫一扫)欢迎添加!