Build a Tool Registry and Policy Engine

A tool registry describes available capabilities; a policy engine decides whether a specific actor may execute a specific call in current context. Keep discovery, model selection, authorization, approval, execution, and verification separate. The model can choose among visible tools, but it cannot grant itself a tool, tenant scope, credential, or exception.

What will you be able to do?

You will define tool and policy contracts, filter the visible tool set, and enforce allow, deny, or approval decisions at execution time.

What should a tool specification contain?

python

from dataclasses import dataclass
from typing import Literal


@dataclass(frozen=True)
class ToolSpec:
    name: str
    version: str
    risk: Literal["read", "write", "destructive", "external"]
    permission: str
    timeout_seconds: int
    idempotent: bool
    requires_approval: bool
    input_schema_hash: str
    output_schema_hash: str

Keep handlers in the registry by internal reference; do not serialize code into model context.

What should policy return?

python

@dataclass(frozen=True)
class PolicyDecision:
    effect: Literal["allow", "deny", "require_approval"]
    reason_code: str
    policy_version: str
    constraints: tuple[str, ...] = ()

Inputs include authenticated identity, tenant, tool/version, normalized arguments, resource attributes, data classification, environment, and current approval. Outputs must be explainable and auditable.

Where should policy run?

Before the run, filter tools that can never apply.
On every proposed call, evaluate exact arguments and current context.
Before commit, re-evaluate stale/changed resources.
After execution, verify result constraints and record evidence.

SDK guardrails, permissions, and hooks can enforce local pieces; the central harness policy keeps business rules consistent across providers.

Failure injection: Change an argument after approval but before execution. The proposal hash and policy engine must reject the call.

How do you prove the policy engine works?

Use table-driven tests for actors, tenants, tools, risk, environments, approval state, and time. Add deny-by-default and unknown-version tests. Run the same policy suite through both provider adapters.

What mistakes should you avoid?

Treating tool visibility as authorization.
Allowing unversioned tool schemas.
Encoding business policy only in provider-specific prompts.
Returning vague deny reasons that operators cannot diagnose.
Allowing policy service failure to default to allow.

Check your understanding

What is the difference between registry and policy?
Why evaluate policy more than once?
Which policy inputs must never come from model claims?

Expert extension

Implement a table-driven policy evaluator and prove with property tests that unknown tools, schemas, tenants, or actors always deny.

Official references

Build a Tool Registry and Policy Engine

What should a tool specification contain?

python

from dataclasses import dataclass
from typing import Literal


@dataclass(frozen=True)
class ToolSpec:
    name: str
    version: str
    risk: Literal["read", "write", "destructive", "external"]
    permission: str
    timeout_seconds: int
    idempotent: bool
    requires_approval: bool
    input_schema_hash: str
    output_schema_hash: str

Keep handlers in the registry by internal reference; do not serialize code into model context.

What should policy return?

python

@dataclass(frozen=True)
class PolicyDecision:
    effect: Literal["allow", "deny", "require_approval"]
    reason_code: str
    policy_version: str
    constraints: tuple[str, ...] = ()

Where should policy run?

Before the run, filter tools that can never apply.

On every proposed call, evaluate exact arguments and current context.

Before commit, re-evaluate stale/changed resources.

After execution, verify result constraints and record evidence.

SDK guardrails, permissions, and hooks can enforce local pieces; the central harness policy keeps business rules consistent across providers.

Failure injection: Change an argument after approval but before execution. The proposal hash and policy engine must reject the call.