AI security philosophy
How I think about AI security
These principles come from the projects. They are Casey’s engineering and security philosophy, not a claim that they are universal law.
01
AI should not determine its own authority
Who determines the next action is the load-bearing question. A model can classify, draft, and recommend. It does not grant itself tools, publication rights, or remediation rights.
Authority is assigned outside the model → Least privilege / separation of duties
Grounded in: Multi-Stage Knowledge Publishing Workflow · Knowledge Publishing Agent · Home-Lab Endpoint Triage Agent
02
High-impact actions require explicit authorization
Isolation, process termination, account disablement, publication, customer packaging, and production deploy are privileged. The agent recommends; a person authorizes.
Human approval gate → Privileged change authorization
Grounded in: Home-Lab Endpoint Triage Agent · Clearward · Basaltborne · Knowledge Publishing Agent
03
Generation and validation are separate concerns
The same model should not certify its own output. Required fields, hashes, schema, RLS, and style checks belong in deterministic code.
Deterministic validators → Verify before trust
Grounded in: Knowledge Publishing Agent · Clearward · Basaltborne · TinyClaw
04
Agents receive only the capabilities they need
Collection, analysis, and remediation use separate authority where practical. Tool allowlists are the capability boundary. Completing a task more easily with an extra tool is a failed control.
Tool allowlist → Authorized capability boundary
Grounded in: Home-Lab Endpoint Triage Agent · Knowledge Publishing Agent · Clearward
05
Actions should be observable and auditable
Log tool calls, inputs, decisions, errors, retries, confidence, approvals, and stopping reasons. If a manipulation attempt is not visible afterward, the control did not hold.
Audit trail → Security logging and accountability
Grounded in: Home-Lab Endpoint Triage Agent · Knowledge Publishing Agent · Basaltborne
06
External data is untrusted input
Logs, documents, websites, filenames, and command output may contain instructions intended to manipulate the model. Tool policy overrides anything asserted in the content.
Content cannot override policy → Untrusted-input handling
Grounded in: Home-Lab Endpoint Triage Agent · Knowledge Publishing Agent · TinyClaw
07
Secrets do not belong in prompts or repositories
Clearward agent instructions forbid collecting passwords, recovery codes, payment-card data, or MFA secrets, and require a human on CAPTCHA, MFA, or payment. Basaltborne never ships service-role credentials to the browser.
Human-only credential boundary → Credential protection
Grounded in: Clearward · Basaltborne
08
Failure modes should fail safely
Stop and escalate when evidence is insufficient, tools fail repeatedly, permissions are exceeded, or confidence requirements are not met. TinyClaw skips a run rather than publish a partial report.
Safe-stop conditions → Fail closed
Grounded in: Home-Lab Endpoint Triage Agent · TinyClaw · Knowledge Publishing Agent
09
Rollback is designed before automation is trusted
Recommended changes include expected results, validation criteria, operational risk, and a rollback procedure. Lab checkpoints and replayable migrations exist so undo is not improvisation.
Checkpoints and planned undo → Recovery / incident response
Grounded in: Home-Lab Endpoint Triage Agent · Basaltborne
10
Human review belongs where consequence and uncertainty justify it
Not every lookup needs a committee. Purchases, safety-relevant vehicle changes, publication, customer release, containment, and production deploy do. Review is placed on consequence, not on ritual.
Review on consequential transitions → Risk-based change control
Grounded in: Domain-Specific AI Expert · Clearward · Basaltborne · Home-Lab Endpoint Triage Agent