Module: KAFKA

How mature is the current KAFKA module? The current role implements a production-grade v1 baseline: dynamic KRaft, full cluster guardrails, cold-start/repair, serial broker admission and dynamic controller join, member retirement (including dead …

The KAFKA module uses two kinds of metric sources, both registered under the same job=kafka scrape job: the JMX target (with a role label) collects the internal state of each JVM, while the protocol exporter target (without a role label) collects the …

Pigsty gives the KAFKA module a unified observability stack that combines metrics, logs, dashboards, and alerts. Monitoring covers both the Kafka JVM internals and the Kafka protocol view, so you never end up seeing only that the process is alive …

The KAFKA module ships two playbooks: kafka.yml deploys an Apache Kafka 4.1+ dynamic KRaft cluster and converges its security, resource, and monitoring state; kafka-rm.yml tears down a cluster or removes a member. Cluster Completeness Constraint …

The KAFKA module installs Kafka under /opt/kafka, manages the service with Systemd, and keeps its persistent intent in pigsty.yml. The files generated on the nodes are not meant to be edited by hand. All of the Kafka CLI examples below use the …

The KAFKA role deliberately exposes only 15 persistent parameters. Details such as topology, listeners, security implementation, storage subdirectories, replication safety, and Exporter placement are derived by the role in a single, consistent way, …

The KAFKA module expresses cluster intent through 15 persistent public parameters; everything else — topology, listeners, storage subdirectories, replication safety, authorization, and Exporter placement — is derived by the role in a single, …

This tutorial starts from a minimal single-node cluster, walking through topic creation and message read/write; it then deploys a separate, secured three-node cluster with application users, ACLs, quotas, and production topics; finally, it …

Kafka is a distributed event-streaming platform. Pigsty’s KAFKA module deploys Apache Kafka 4.1+ dynamic KRaft clusters on managed nodes from RPM/DEB packages, with unified management of security, resources, lifecycle, and observability. Current …