Database observability connects user outcomes to PostgreSQL workload, latency, errors, waits, saturation, replication, storage, maintenance, and recovery readiness. An SLO defines a measured promise; alerts should signal exhausted error budget or imminent risk and point to a runbook. Logs and traces must preserve correlation without exposing SQL parameters, secrets, or tenant content.
Use four views: demand (queries/transactions), outcomes (latency/errors), resources (CPU/I/O/connections/disk), and internal work (locks/WAL/checkpoints/vacuum/replication). A dashboard without a decision is decoration.
pg_stat_activity, cumulative statistics views, logs, and commonly pg_stat_statements provide evidence. Extension availability and configuration vary.
Example SLO: “99.9% of authorized ticket-detail requests complete successfully under 400 ms over 28 days, excluding declared maintenance.” Database query time is one component, not the entire SLI.
Trace retrieval SQL, candidate counts, reranking, model request, validation, and tool execution as distinct spans. Version prompts, embedding models, indexes, and datasets. Redact content by default. Monitor grounded answer success and authorization failures alongside latency and cost—not instead of database health.
Trigger safe known conditions in learning: statement timeout, constraint failure, and pool saturation. Verify signal, correlation, redaction, alert threshold, and runbook action. Confirm an operator can distinguish application latency from database wait and model latency.
Create an observability specification for “open a ticket and generate a cited draft reply.”
Acceptance criterion: spans and metrics locate DB, retrieval, model, and validation time; no prompt/customer body appears in default telemetry.