AI agent loop checklist: validation steps before autonomous code ships

The first time an AI agent pushes untested code straight to main, you learn exactly why guardrails exist. The build fails in production, a deprecated API call slips through, or—worse—the diff looked clean but silently broke an integration three modules away. Autonomous coding workflows promise velocity, but they collapse without systematic validation at every handoff in the loop.

This checklist walks through the validation gates worth enforcing when you let an agent write, edit, and commit code with minimal human oversight. These aren't nice-to-haves; they're the mechanical stops that prevent "fast iteration" from becoming "fast accumulation of defects."

Pre-edit constraint validation

Before the agent touches a file, validate that it understands the boundaries of the task. Constraint validation means checking that the agent's plan respects the invariants of your codebase—type signatures, module boundaries, deprecated patterns, architectural decisions you've already made.

What to check:

• Scope limits. If the task is "fix the login timeout," the agent shouldn't be proposing edits to the payment handler. Parse the plan and confirm file paths fall within the expected surface area.
• Schema compatibility. If your API contract is versioned or you maintain backward compatibility guarantees, validate that the proposed changes don't alter public interfaces without explicit intent.
• Banned patterns. Maintain a list of deprecated functions, risky language features, or anti-patterns your team has moved away from. Reject plans that reintroduce eval(), global mutable state, or whatever your codebase has outgrown.

In Goatfied's loop, constraints are applied before the edit step. The agent generates a plan, we run static checks against that plan's scope, and only then do we allow file modifications. This catches category errors early—like an agent deciding to refactor the entire auth module when you asked it to add a logging statement.

# Example constraint file fragment

constraints:

  scope:

    - "src/auth/**"

    - "!src/auth/legacy/**"

  banned_imports:

    - "lodash.merge"  # we migrated to structuredClone

  api_stability: "public_interfaces_immutable"

If the plan violates a constraint, the agent doesn't get to edit. It either revises the plan or the task is flagged for human review.

Compile-time validation

Code that doesn't compile is code that doesn't ship. This should be obvious, yet many early agent workflows skipped this gate because "the diff looked reasonable." Compilation is the cheapest, fastest proof that the agent produced syntactically and type-correct code.

What to check:

• Language-specific compilation. TypeScript's tsc --noEmit, Go's go build, Rust's cargo check—run the relevant compiler with strict settings. No warnings-as-noise here; warnings in automated workflows are defects you haven't fixed yet.
• Dependency resolution. Did the agent add an import that doesn't exist? Did it assume a function signature that changed three commits ago? Compilation will catch it if your tooling is wired correctly.
• Incremental checks. If you're editing a large monorepo, validate only the changed modules and their direct dependents. Full rebuilds on every edit will throttle the loop.

Goatfied runs compile checks immediately after the edit step and before any commit. If the build fails, the agent receives the compiler output as feedback and retries with a corrected diff. This tight loop—edit, compile, retry—means most type errors get resolved in seconds rather than blocking CI thirty minutes later.

Lint and format enforcement

Linters catch what compilers miss: style violations, code smells, complexity hotspots, and subtle bugs that are syntactically legal but semantically questionable.

What to check:

• Autofix-capable rules first. Run eslint --fix, black, gofmt, or equivalent. If the linter can correct the code automatically, let it. The agent's output doesn't need to be perfect on spacing; it needs to be correctable without human intervention.
• Hard-fail rules second. Unused variables, unreachable code, security lint warnings (like eslint-plugin-security flagging innerHTML usage)—these should block the commit.
• Complexity metrics. If your linter can measure cyclomatic complexity or nesting depth, set thresholds. An agent that generates a 300-line function with nine levels of indentation has misunderstood the task.

When lint fails, treat it the same way as a compile failure: feed the errors back to the agent and retry. Most linters produce machine-readable output (JSON or SARIF), which makes this loop straightforward to automate.

# Example validation sequence

tsc --noEmit && eslint --max-warnings 0 src/

If either step exits non-zero, the edit is rejected and the agent sees the error details in the next iteration.

Unit test execution

Tests are your functional contract. If the agent's edit breaks a test, it broke something that used to work—or it misunderstood what "working" means.

What to check:

• Run affected tests only. Use test impact analysis or workspace-aware tooling (jest --findRelatedTests, go test ./... scoped to changed packages) to avoid re-running your entire suite on every iteration.
• Fail fast. Configure test runners to stop on first failure. The agent doesn't need to see all 47 broken assertions; it needs to see which invariant it violated and adjust.
• Coverage deltas. If your workflow allows, check whether the edit reduced coverage. An agent that deletes a function without removing its tests is fine; one that removes tests to make the build pass is not.

In Goatfied's loop, unit tests run after lint passes and before any version control operation. The agent can iterate on edits multiple times—adjusting logic, refining mocks, fixing off-by-one errors—until the test suite is green. This happens entirely in the workspace; nothing hits CI until validation is complete.

Integration and contract testing

Unit tests validate individual functions. Integration tests validate that the system still behaves correctly when those functions interact with databases, APIs, message queues, or other services.

What to check:

• API contract tests. If your service exposes an HTTP API or gRPC interface, run contract tests (Pact, Dredd, or snapshot-based tools) to confirm the agent didn't change response shapes or status codes.
• Database migrations. If the agent modified schema or query logic, validate migrations against a test database. Does the migration apply cleanly? Does the rollback work?
• Stub external dependencies. Use Docker Compose, Testcontainers, or in-memory fakes so integration tests remain fast and deterministic. An agent loop that waits five minutes for a staging environment to spin up will never iterate quickly enough to be useful.

Not every task needs full integration testing—fixing a typo in a README does not. But if the agent touched business logic, data access, or inter-service communication, integration tests should be a mandatory gate before the code can merge.

Security and compliance scans

Automated security tooling catches classes of vulnerabilities that compilers and linters ignore: dependency CVEs, hardcoded secrets, insecure deserialization, SQL injection risks.

What to check:

• Dependency scanning. Run npm audit, go mod verify, or cargo audit. If the agent introduced a new dependency, validate it's not flagged for known vulnerabilities.
• Secret detection. Tools like truffleHog, detect-secrets, or gitleaks scan diffs for API keys, tokens, private keys. This should be non-negotiable in any autonomous workflow.
• Static analysis for vulnerability patterns. Semgrep, CodeQL, or language-specific tools (Bandit for Python, Brakeman for Rails) can identify risky patterns before they reach production.

Security scans should run after functional tests pass. If a vulnerability is detected, the agent either fixes it in the retry loop or the task escalates to human review. There's no scenario where "the tests passed but we shipped a SQL injection" is acceptable.

Diff review and change size limits

Even when all automated checks pass, a 3,000-line diff from an agent is a red flag. Large, sprawling changes are hard to audit, hard to revert, and often indicate the agent misunderstood scope.

What to check:

• Lines changed threshold. Set a hard limit—maybe 200 lines, maybe 500, depending on your team's tolerance. If the agent exceeds it, split the task or require human sign-off.
• Files changed count. If the agent modified 30 files for a "fix broken link" task, something went wrong.
• Semantic diff review. Tools like difftastic or AST-aware differ can highlight meaningful changes versus formatting churn. Prioritize reviewing logic changes over whitespace.

Goatfied enforces small, reversible diffs by design. The agent loop is optimized for narrow, well-scoped edits—closer to "fix this function" than "refactor this module." When changes stay small, both automated validation and human review become tractable.

Pre-merge human review gates

Some decisions still require human judgment: architectural direction, user-facing copy, changes that affect SLAs or cost. Define clear gates where automation pauses and a human must approve.

What to check:

• High-risk paths. Authentication logic, payment processing, data deletion—flag these for mandatory review regardless of test status.
• Policy-driven checks. If your compliance framework requires sign-off on schema changes or API surface alterations, enforce it here.
• Audit trails. Log every validation step, every retry, every gate passed. If you're in a regulated industry, you need to demonstrate why autonomous code was allowed to ship.

The goal isn't to block every agent-generated commit with human review—that defeats the purpose of automation. The goal is to escalate only the changes that carry risk your automated checks can't capture.

Putting the checklist into practice

A realistic validation sequence for an autonomous agent loop looks like this:

1. Plan validation → constraints checked, scope approved

2. Edit → agent modifies files

3. Compile → tsc, cargo check, or equivalent

4. Lint → autofix + hard-fail rules

5. Unit tests → scoped to affected code

6. Integration tests → if business logic changed

7. Security scans → secrets, CVEs, vulnerability patterns

8. Diff review → size limits, semantic checks

9. Human gate → if high-risk or policy-mandated

10. Commit + push → only when all gates pass

Each failed gate sends feedback to the agent for retry. Each passed gate increases confidence that the code is safe to ship.

If you're building this yourself, start with steps 1–5. Compile + lint + unit tests will catch 80% of defects. Add integration and security scanning as your automation matures. Add human gates based on actual incidents, not theoretical risk.

If you're using Goatfied, most of this sequence is built into the loop: plan → constrain → edit → validate → retry. You configure the constraints, point at your test commands, and the loop enforces the gates automatically. Self-hosted or managed, the validation steps remain consistent.

Related reading

• The Goatfied agent loop: how we ship code that actually compiles first try
• Why AI agents fail on the second edit, and how validation loops fix it