Pair nodes, manage shared configuration, and optionally replicate SQLite data.

Trestle has two deliberately separate distributed surfaces: Gantry clustering establishes authenticated peer relationships for operations and configuration propagation; SQLite replication uses Raft for authoritative collection and record state. Pairing a node never silently makes it a Raft voter.

Choose the surface you need

Gantry peer clustering

Each installation has its own node identity. Pairing is explicit and invitation based: initialize the identity, create a one-time invitation on one node, begin the join from the other, approve it, then collect the approval. Production public endpoints use HTTPS; plaintext peer transport is only available through the explicit insecure-development setting.

trestle cluster init --name node-a --endpoint https://a.example.com
trestle cluster invite --secret-file /tmp/trestle-invite
trestle cluster join --url https://a.example.com --token-file /tmp/trestle-invite
trestle cluster joins
trestle cluster approve --id <join-id>
trestle cluster collect --id <outbound-join-id>

Cluster administration

trestle cluster members
trestle cluster status --target all
trestle cluster status --target local
trestle cluster status --target members
trestle cluster status --target node:<node-id>
trestle cluster rotate --id <node-id> --secret-file /secure/new-credential
trestle cluster revoke --id <node-id>
trestle cluster remove --id <node-id>

trestle cluster reject --id <join-id> rejects a pending join. Cluster actions are recorded in the cluster audit history.

Management UI and API

The management application includes a Cluster view for identity, members, pending and outbound joins, status, audit history and propagation. The administrator API exposes identity, members, invitations, joins, outbound joins, summary, comparison, audit and propagation under /admin/v1/cluster/.... Peer RPC lives under /api/cluster/v1/... and uses cluster credentials rather than application-user credentials.

Compare and propagate configuration

Trestle can compare supported product configuration across paired nodes and propagate selected configuration through the shared Gantry propagation contract. Propagation supports preview/dry-run before apply, target selection, durable profiles and reconciliation history. Trestle exports validated configuration envelopes and applies them through product semantics rather than copying raw database rows.

Propagation is not database replication. It distributes supported configuration; it does not copy arbitrary records, credentials or node identity.

SQLite Raft replication

Replication is optional and separate from Gantry pairing. It replicates collection definitions and records as semantic mutations through Raft. Authentication/session state, credentials, files, jobs, webhook and function execution, audit/events and other external side effects are not silently replayed. PostgreSQL replication fails closed rather than presenting a partial clustered mode.

TRESTLE_REPLICATION_ENABLED=true
TRESTLE_REPLICATION_NODE_ID=node-a
TRESTLE_REPLICATION_LISTEN=10.0.0.10:7440
TRESTLE_REPLICATION_TLS_CERT=/etc/trestle/node-a.crt
TRESTLE_REPLICATION_TLS_KEY=/etc/trestle/node-a.key
TRESTLE_REPLICATION_TLS_CA=/etc/trestle/replication-ca.crt

Production replication requires the complete certificate/key/CA triplet. Plaintext is an explicit local-development opt-in.

Form a replicated cluster

1. Configure SQLite replication and stable node IDs/addresses on every node.
2. Pair every participant through trestle cluster ... so peers authenticate in both directions.
3. Bootstrap one node with TRESTLE_REPLICATION_BOOTSTRAP=true.
4. Verify the leader with trestle replicate status.
5. Add each remaining voter explicitly with trestle replicate join <node-id> <raft-address>.
6. Verify voter membership and readiness from the cluster.

Gantry membership and Raft voter membership remain separate throughout this process.

Replication operations

trestle replicate status
trestle replicate join <node-id> <raft-address>
trestle replicate snapshot

Cluster and replication commands must use the same Trestle environment as the running service. On a systemd installation, load the configured service environment so the CLI targets the correct data directory and database.

Quorum and follower writes

3/3  - normal operation
2/3  - quorum remains; authoritative writes continue
1/3  - authoritative mutations fail closed; no local SQLite fallback

Mutations may be submitted to a healthy ready follower. It authenticates the request, forwards the semantic mutation to the leader, waits for the committed index to apply locally, then returns. A catching-up follower can transiently return replication unavailable (503); the mutation is not invisibly committed.

Durability, snapshots and recovery

<data-dir>/
  trestle.db
  replication/
    raft.db
    snapshots/

The SQLite product database, Raft log and snapshots are all meaningful durable state. Restarts, whole-cluster restarts and offline-follower catch-up rebuild through committed Raft state. trestle replicate snapshot forces a snapshot. Do not manually copy, edit or delete Raft state as an upgrade procedure.

Idempotency and preconditions

Caller idempotency is consensus-owned: the same Idempotency-Key and semantic intent replay the original committed result on any node; reusing a key for different content is a 409 conflict. Record updates and deletes use If-Match; stale versions are caller conflicts and never silently overwrite newer state.

Backup and restore boundary

Portable backups intentionally exclude cluster identity, private signing keys, membership credentials and topology. A restored node generates a new identity and must be paired again. Restore into a destination that already contains cluster identity or membership fails closed rather than merging trust state.

Security and failure boundary

Disabling or revoking a Gantry peer stops cluster authorization in both directions; Raft membership is a separate operator decision. Credential rotation changes the local outbound credential and version. Production Raft transport uses mutual TLS. The replicated data surface remains collections and records only; files, credentials, sessions, jobs, integrations, audit/events and other side effects remain node-local unless a separate distributed contract explicitly says otherwise.