# demo/kafka

> 四节点 dynamic KRaft 示例：单节点明文开发集群与三节点 TLS/SCRAM 高可用基线

---

LLMS 索引： [llms.txt](/zh/llms.txt)

---

`demo/kafka` 在四个节点上声明两套 Kafka 4.x dynamic KRaft 集群：单节点明文开发集群 `kf-meta`，以及三节点 TLS/SCRAM/ACL 演示集群 `kf-test`。


--------

## 配置概览

- 配置名称：`demo/kafka`
- 节点数量：4 个
- `kf-meta`：单节点 combined Broker/Controller，明文模式
- `kf-test`：3 个 combined 节点，TLS/SCRAM/ACL，Topic 副本数 3、`min.insync.replicas=2`
- 模块状态：KAFKA BETA

```bash
./configure -c demo/kafka -s
./deploy.yml
./kafka.yml -l kf-meta
./kafka.yml -l kf-test
```

`deploy.yml` 只部署核心链路，并不会自动执行 KAFKA 剧本。每次 `kafka.yml` 运行都应选择一个完整的 Kafka 集群；角色会拒绝只选中部分成员的收敛操作。


--------

## 配置内容

源文件地址：[`pigsty/conf/demo/kafka.yml`](https://github.com/pgsty/pigsty/blob/main/conf/demo/kafka.yml)

```yaml
---
#==============================================================#
# File      :   kafka.yml
# Desc      :   pigsty: 4 node kafka demo (dynamic KRaft)
# Ctime     :   2026-07-17
# Mtime     :   2026-07-17
# Docs      :   https://pigsty.io/docs/kafka
# License   :   Apache-2.0 @ https://pigsty.io/docs/about/license/
# Copyright :   2018-2026  Ruohang Feng / Vonng (rh@vonng.com)
#==============================================================#

# One pass installation with:
# ./deploy.yml
# ./kafka.yml -l kf-main
# ./kafka.yml -l kf-test
#==============================================================#
# 1.  kf-meta-1 @ 10.10.10.10:9092   single-node dev cluster (plaintext)
# 2.  kf-test-1 @ 10.10.10.11:9092 \
# 3.  kf-test-2 @ 10.10.10.12:9092 --- 3-node secure HA demo baseline (scram)
# 4.  kf-test-3 @ 10.10.10.13:9092 /   dynamic KRaft, TLS/SCRAM/ACL, RF=3/minISR=2
#==============================================================#
# kafka clients are cluster-aware and connect to every broker directly:
# bootstrap with e.g. 10.10.10.11:9092,10.10.10.12:9092,10.10.10.13:9092


all:
  children:

    # infra cluster for proxy, monitor, alert, etc..
    infra: { hosts: { 10.10.10.10: { infra_seq: 1 } } }

    # single-node kafka dev cluster: combined broker/controller, plaintext
    kf-meta:
      hosts:
        10.10.10.10: { kafka_seq: 1 }
      vars:
        kafka_cluster: kf-meta
        kafka_topics:
          - { name: quickstart.events ,partitions: 1 ,replication_factor: 1 ,config: { retention.ms: 86400000 } }

    # 3-node secure HA demo baseline: dynamic KRaft, TLS/SCRAM/ACL, RF=3/minISR=2
    kf-test:
      hosts:
        10.10.10.11: { kafka_seq: 1 }
        10.10.10.12: { kafka_seq: 2 }
        10.10.10.13: { kafka_seq: 3 }
      vars:
        kafka_cluster: kf-test
        kafka_security: scram
        kafka_heap_opts: '-Xms512M -Xmx512M' # 2GiB demo nodes cannot safely spare the 1GiB production default
        kafka_users:               # app principal with prefixed topic/group acls
          - name: test-app
            password: KafkaApp.Test
            acls:
              - { resource: topic   ,name: 'test.'       ,pattern: prefixed ,operations: [ Read, Write, Describe ] }
              - { resource: group   ,name: 'test.'       ,pattern: prefixed ,operations: [ Read ] }
              - { resource: cluster ,name: kafka-cluster ,operations: [ Describe, IdempotentWrite ] }
        kafka_topics:
          - name: test.events
            partitions: 3
            replication_factor: 3
            config: { min.insync.replicas: 2 ,cleanup.policy: delete ,retention.ms: 604800000 }

  vars:
    version: v4.5.0                   # pigsty version string
    admin_ip: 10.10.10.10             # admin node ip address
    region: default                   # upstream mirror region: default|china|europe
    infra_portal:                     # infra services exposed via portal
      home : { domain: i.pigsty }     # default domain name

    # kafka & java packages are required in the local repo for the kafka module (if using local repo)
    repo_extra_packages: [ kafka-stack ,java-runtime ]

    #----------------------------------------------#
    # PASSWORD : https://pigsty.io/docs/setup/security/
    #----------------------------------------------#
    grafana_admin_password: pigsty
    grafana_view_password: DBUser.Viewer
...
```


--------

## 配置解读

- `kf-meta` 创建 `quickstart.events`，用于单机开发与连通性测试。
- `kf-test` 创建 `test-app` SCRAM 用户、前缀 ACL 与 `test.events` 三副本 Topic。
- 在线安装时由平台映射安装 `kafka-stack` 与 `java-runtime`；若只使用本地仓库，必须先把这两个包组完整纳入仓库。
- 模板中的地址和密码均为演示值，部署前应按实际拓扑与安全要求修改。

更多操作、安全与扩缩容约束参见 [KAFKA 模块](/docs/kafka/)。