A production prompt is a versioned interface between software and a probabilistic model. It defines role, task, trusted context, untrusted input boundaries, output schema, constraints, examples, and failure behavior. Reliability comes from structured output, validation, evaluation, and controlled retries—not from adding more confident wording.
Method note: Prompt contracts, schemas, routing rules, and tests are application design artifacts rather than equivalent Azure resource-management operations. Microsoft Foundry portal and SDK deployment choices are covered in the preceding chapter; this chapter keeps the shared engineering rules visible together.
Do not place authorization solely in the prompt. Enforce it in code before retrieval and tool execution.
Free text is difficult to validate. A JSON schema turns the model response into a typed boundary.
Use the current model/API's structured-output capability where supported. Parse, validate, and reject invalid responses. Do not silently coerce a dangerous value into the expected type.
Retry transport throttling and transient service errors with exponential backoff and jitter. A schema failure may justify one constrained repair attempt. Do not repeatedly resubmit the same unsafe or impossible task. Cap attempts, tokens, time, and total cost; then fail visibly or route to a human.
Routing selects a model/deployment based on task difficulty, modality, latency, risk, region, and cost. A deterministic classifier can route simple extraction to a small model and complex synthesis to a capable one. Measure the router as part of the system; a cheap incorrect route can cost more through failures and retries.
Microsoft Foundry can expose model routing capabilities, while your application may also implement explicit routing. Keep an evaluation record for every route and fallback.
Measure input tokens, cached input where offered, output tokens, tool calls, retries, and embedding/reranking work. Reduce cost safely by:
Do not remove context that protects correctness merely to lower token count. Optimize cost per successful outcome, not cost per call.
Store prompt templates and schemas in source control or an approved prompt registry. Record version, model deployment, parameters, evaluator set, author, and change reason. Promote a prompt only after regression evaluation. Avoid editing a production prompt directly in a portal without a traceable review.
Use semantic evaluators cautiously and calibrate them against human judgments. A model grading another model is a signal, not ground truth.