Categories: Task

Citus is a PostgreSQL extension that transforms PostgreSQL into a distributed database, enabling horizontal scaling across multiple nodes to handle large amounts of data and queries. Patroni v3.0+ provides native high-availability support for Citus, …

If a classic 3-node HA deployment experiences simultaneous failure of two nodes (majority), the system typically cannot complete automatic failover and requires manual intervention. First, assess the status of the other two servers. If they can be …

The KAFKA module ships two playbooks: kafka.yml deploys an Apache Kafka 4.1+ dynamic KRaft cluster and converges its security, resource, and monitoring state; kafka-rm.yml tears down a cluster or removes a member. Cluster Completeness Constraint …

The KAFKA module installs Kafka under /opt/kafka, manages the service with Systemd, and keeps its persistent intent in pigsty.yml. The files generated on the nodes are not meant to be edited by hand. All of the Kafka CLI examples below use the …

Service Management systemctl status code-server systemctl restart code-server systemctl status jupyter systemctl restart jupyter Logs: journalctl -u code-server -f journalctl -u jupyter -f Workspace and Context vibe_dir creates these under …

VIBE provides the vibe.yml playbook to deploy Code-Server, JupyterLab, Node.js, Claude Code, and Codex CLI. vibe.yml includes only node_id and vibe roles, it does not include node/infra. Run deploy.yml first, or explicitly run node.yml and infra.yml. …

Common operations: Initialize Instance Reconfigure Remove Instance Add New Instance Shared Mount Across Nodes PITR Recovery Troubleshooting Performance Tuning See FAQ for more. Initialize Instance ./juice.yml -l <host> ./juice.yml -l <host> -e …

JUICE module provides juice.yml playbook to deploy and remove JuiceFS instances. juice.yml Task structure in juice.yml: juice_id : validate config, check port conflicts juice_install : install juicefs package juice_cache : create shared cache dir …

The Docker module provides a default playbook docker.yml for installing Docker Daemon and Docker Compose. docker.yml Playbook source file: docker.yml Running this playbook will install docker-ce and docker-compose-plugin on target nodes with the …

You can use PostgreSQL as Grafana’s backend database. This is a great opportunity to understand Pigsty’s deployment system. By completing this tutorial, you’ll learn: How to create a new database cluster How to create new business users in an …

Here are some common Redis administration task SOPs (Standard Operating Procedures): The REDIS module defaults to redis_type: redis. With redis_type: valkey, the server and client commands become valkey-server and valkey-cli. Examples on this page …

The REDIS module provides two playbooks for deploying/removing Redis clusters/nodes/instances: redis.yml: Deploy Redis cluster/node/instance redis-rm.yml: Remove Redis cluster/node/instance redis.yml The redis.yml playbook for deploying Redis …

Create Cluster To create a cluster, define it in the config inventory and run the minio.yml playbook. minio: { hosts: { 10.10.10.10: { minio_seq: 1 } }, vars: { minio_cluster: minio, minio_type: silo } } The configuration above defines an SNSD …

The MINIO module provides two built-in playbooks: minio.yml: Install and configure Silo minio-rm.yml: Remove Silo, its configuration, and optionally its data minio.yml minio.yml runs with hosts: all, but its pre-tasks skip hosts where minio_cluster …

The ETCD module provides two core playbooks: etcd.yml for installing and configuring etcd clusters, and etcd-rm.yml for removing etcd clusters or members. Architecture Change: Pigsty v3.6+ Since Pigsty v3.6, the etcd.yml playbook focuses on cluster …

Common etcd admin SOPs: Create Cluster: Initialize an etcd cluster Destroy Cluster: Destroy an etcd cluster CLI Environment: Configure etcd client to access server cluster RBAC Authentication: Use etcd RBAC auth Reload Config: Update etcd server …

Here are common administration operations for the NODE module: Add Node Remove Node Create Admin Bind VIP Add Node Monitoring Other Tasks For more questions, see FAQ: NODE Add Node To add a node to Pigsty, you need passwordless ssh/sudo access to …

Pigsty provides two playbooks related to the NODE module: node.yml: Add nodes to Pigsty and configure them to the desired state node-rm.yml: Remove managed nodes from Pigsty Two wrapper scripts are also provided: bin/node-add and bin/node-rm, for …

Pigsty maintains a self-signed Certificate Authority (CA) on the admin node by default. It signs certificates for PostgreSQL, Patroni, etcd, Silo, Nginx, and other internal services. Public Nginx entries can use Certbot/Let’s Encrypt certificates …

This document covers daily management operations for the INFRA module, including installation, uninstallation, scaling, and component maintenance. Install INFRA Module Use the infra.yml playbook to install the INFRA module on the infra group: …

Use domain names instead of IP addresses to access Pigsty’s various web services. Quick Start Add the following static resolution records to /etc/hosts: 10.10.10.10 i.pigsty Replace IP address with your actual Pigsty node’s IP. Why Use Domain Names …

Pigsty’s REPO role downloads required packages and creates a local YUM/APT repository under /www/pigsty that Nginx can serve. The current package candidate is SOW 0.3.0; the source uses SOW to generate metadata for both repository types instead of …

Pigsty installs Nginx on INFRA nodes as the entry point for all web services, listening on standard ports 80/443. In Pigsty, you can configure Nginx to provide various services through inventory: Expose web interfaces for monitoring components like …

This section covers daily administration and operations for Pigsty deployments.

Pigsty provides three playbooks related to the INFRA module: deploy.yml: Deploy the NODE, INFRA, ETCD, MINIO, and PGSQL core modules on all nodes in one pass infra.yml: Initialize Pigsty infrastructure on infra nodes infra-rm.yml: Remove …

You can define an OPTIONAL L2 VIP on a PostgreSQL cluster, provided that all nodes in the cluster are in the same L2 network. This VIP works on Master-Backup mode and always points to the node where the primary instance of the database cluster is …

Pigsty provides a series of playbooks for cluster provisioning, scaling, user/database management, monitoring, backup & recovery, and migration. Playbook Function pgsql.yml Initialize PostgreSQL cluster or add new replicas pgsql-rm.yml Remove …

Accidental Data Deletion If it’s a small-scale DELETE misoperation, you can consider using the pg_surgery or pg_dirtyread extension for in-place surgical recovery. -- Immediately disable Auto Vacuum on this table and abort Auto Vacuum worker …

Pigsty v4.5.0 provides two local shell utilities: pg-fork copies a PostgreSQL data directory and gives the copy a separate port. pg-pitr invokes pgBackRest to restore a stopped data directory to a selected target. They are useful for sandbox drills, …

Use node_hugepage_count and node_hugepage_ratio or /pg/bin/pg-tune-hugepage If you plan to enable HugePages, consider using node_hugepage_count and node_hugepage_ratio, and apply with ./node.yml -t node_tune. HugePages have pros and cons for …

This tutorial drills PostgreSQL point-in-time recovery in Pigsty v4.5.0’s four-node sandbox. The main path runs pgsql-pitr.yml as down → pitr → up, giving the operator a separate validation gate before data overwrite, timeline promotion, and HA …

Run backup commands as the database superuser (pg_dbsu, postgres by default) on a database node. You can use any of these entry points: pig pb: the PIG CLI wrapper, with automatic stanza detection, DBSU switching, and safety checks; this is the …

Two parameters decide where backups are stored: pgbackrest_repo defines candidate repositories, while pgbackrest_method selects one. Repository keys are rendered deterministically as pgBackRest repo1-* options, so any supported pgBackRest repository …

Pigsty’s backup and restore operations ultimately execute pgBackRest commands. Using them safely requires both pgBackRest’s model and the mapping from Pigsty’s orchestration layers to native options. Core pgBackRest Concepts Stanza: the Cluster’s …

Pigsty includes a built-in playbook pgsql-migration.yml that implements online database migration based on logical replication. With pre-generated automation scripts, application downtime can be reduced to just a few seconds. However, note that …

This document lists potential failures in PostgreSQL and Pigsty, as well as SOPs for locating, handling, and analyzing issues. Disk Space Exhausted Disk space exhaustion is the most common type of failure. Symptoms When the disk space where the …

Pigsty uses pgBackRest for PostgreSQL backups. It supports full, differential, and incremental backups, parallel processing, encryption, and Silo/S3 object storage. Every PGSQL cluster is configured for backup and WAL archiving by default. This …

Cloning is one of the safest and most useful applications of recovery: leave production untouched and restore its historical state into another cluster. You can recover accidentally deleted data from the clone, validate backups in a drill, inspect a …

Pigsty provides three restore entry points. They share the same parameter semantics, but serve different scopes: Entry point Use case What it controls pgsql-pitr.yml Production cluster recovery HA pause, multiple nodes, etcd cleanup, restore, and …

The chart below combines the “Recovery Window” and “Backup Storage Usage” on a single timeline (0~108h) so they can be inspected together. Under the same assumptions (database size 100GB, daily writes 10GB), it shows how both metrics evolve over 30 …

Quick Start PostgreSQL version upgrades fall into two types: minor version upgrade and major version upgrade, with very different risk and complexity. Type Example Downtime Data Compatibility Risk Minor upgrade 17.2 → 17.3 Seconds (rolling) Fully …

Quick Start Pigsty provides 575 extensions. Using extensions involves four steps: Download, Install, Configure, Enable. pg-meta: hosts: { 10.10.10.10: { pg_seq: 1, pg_role: primary } } vars: pg_cluster: pg-meta pg_extensions: [ postgis, timescaledb, …

Pigsty uses crontab to manage scheduled tasks for routine backups, freezing aging transactions, and reorganizing bloated tables and indexes. Quick Reference Operation Quick Command Description Configure Cron Jobs ./pgsql.yml -t pg_crontab -l <cls> …

Overview Pigsty’s PGSQL module consists of multiple components, each running as a systemd service on nodes. (pgbackrest is an exception) Understanding these components and their management is essential for maintaining production PostgreSQL clusters. …

Overview Pigsty uses Pgbouncer as PostgreSQL connection pooling middleware, listening on port 6432 by default, proxying access to local PostgreSQL on port 5432. This is an optional component. If you don’t have massive connections or need transaction …

Quick Start Pigsty uses declarative management: first define HBA rules in the inventory, then use bin/pgsql-hba <cls> to refresh. pg-meta: hosts: { 10.10.10.10: { pg_seq: 1, pg_role: primary } } vars: pg_cluster: pg-meta pg_hba_rules: # <--- Define …

Overview Pigsty uses Patroni to manage PostgreSQL clusters. It handles config changes, status checks, switchover, restart, reinit replicas, and more. To use Patroni for management, you need one of the following identities: From INFRA node as admin …

Quick Start Pigsty uses declarative management: first define databases in the inventory, then use bin/pgsql-db <cls> <dbname> to create or modify. pg-meta: hosts: { 10.10.10.10: { pg_seq: 1, pg_role: primary } } vars: pg_cluster: pg-meta …

Quick Start Pigsty uses declarative management: first define users in the inventory, then use bin/pgsql-user <cls> <username> to create or modify. pg-meta: hosts: { 10.10.10.10: { pg_seq: 1, pg_role: primary } } vars: pg_cluster: pg-meta pg_users: [{ …

Quick Reference Action Command Description Create Cluster bin/pgsql-add <cls> Create a new PostgreSQL cluster Expand Cluster bin/pgsql-add <cls> <ip...> Add replica to existing cluster Shrink Cluster bin/pgsql-rm <cls> <ip...> Remove instance from …