Agent reliability is the probability of producing a verified, policy-compliant outcome within time and cost objectives—not merely receiving a model response. Scale through admission control, queues, bounded concurrency, isolation, caching, and backpressure. Optimize after tracing the critical path, and degrade by reducing optional work rather than weakening safety, evidence, or authorization.
You will define SLOs, capacity controls, and an optimization sequence for SupportOps.
Measure distributions (p50/p95/p99), not only averages.
Use per-tenant quotas, global admission control, priority queues, worker pools by tool/risk profile, provider concurrency limits, and backpressure. Autoscale on queue age, active leases, provider limits, and resource saturation. Prevent one open-ended tenant workload from starving short safe tasks.
Never cache authorization decisions beyond their valid context.
Track model, tool, retrieval, compute, storage, and human review costs against verified outcome. Budgets, smaller context, better routing, fewer retries, tool search, and early deterministic rejection often save more than choosing a cheaper model blindly.
Failure injection: Rate-limit the preferred provider and saturate one worker pool. The system should queue, route only to eligible fallbacks, and protect high-priority work without disabling policy.
Load-test normal and burst traffic; inject provider latency, tool errors, queue duplication, storage failure, and approval delay. Verify SLOs, backpressure, fairness, and recovery.
Build a simple capacity model from arrival rate, average service time, concurrency, and provider quotas. Define admission and shedding rules before running a load test.