agent-loop
Completion ranking with repository signals
Learn how AI coding agents use repository-wide signals like pull requests and test outcomes to rank code completions better than traditional syntax-based methods.
Modern code editors powered by AI generate dozens—sometimes hundreds—of suggestions for every keystroke. When you type a function name, the completion engine races through possible endings: method calls, variable names, imports, boilerplate. The challenge isn't generating candidates; it's ranking them so the best option appears first. Traditional approaches use frequency counts and local scope, but agent-loop architectures add a critical dimension: repository-wide signals that reflect how your team actually codes.
An agent loop is a continuous feedback cycle where AI coding agents propose changes, test them in real environments, observe the results, then refine their next proposal based on what worked. Instead of ranking completions purely on syntax patterns or token probability, agent-loop systems mine your codebase history—pull requests, test outcomes, code review feedback, runtime telemetry—to surface suggestions that align with your conventions, preferred libraries and proven patterns.
Why static ranking falls short
Most autocomplete systems rank suggestions using:
- Language model confidence: how probable the next tokens are given the prompt
- Lexical proximity: symbols already in scope or recently used
- Popularity heuristics: frequency of a pattern in public open-source data
These work for generic code, but they ignore your repository's unique conventions. If your team wraps every database call in a custom retry decorator, a static ranker won't know that. If you've deprecated a utility function in favor of a new one, the old function may still rank higher because it appears more often in training data. Static ranking treats every codebase as interchangeable.
Repository signals that improve ranking
Agent-loop systems incorporate signals from the living, changing state of your repository:
| Signal | What it captures | Example |
|--------|------------------|---------|
| Merge frequency | Code patterns that passed review and landed | A custom error handler merged in 47 PRs ranks above a one-off variant |
| Test pass rate | Completions that produce working code | An import path that breaks tests 60% of the time drops in rank |
| Review comments | Explicit team preferences | A linter disable comment repeatedly flagged in reviews signals "avoid this pattern" |
| Runtime telemetry | Performance and error rates in production | A library function with high latency appears with a warning annotation |
| Recency weighting | Newer patterns reflect current standards | The new logging library introduced last quarter outranks the legacy one |
| Author proximity | Code style of the person typing | If you always use arrow functions, those rank above function declarations |
The feedback loop is simple: the agent proposes a completion, the developer accepts or rejects it, the code is committed and tested, and the outcome updates the ranking model. Over time, the autocomplete learns your codebase's idioms as if a senior engineer were whispering in its ear.
How the agent loop collects and applies feedback
Here's the cycle in practice:
1. Proposal: you type const result = fetch, and the agent offers three completions—fetchUser(), fetchWithRetry(), fetch(url).
2. Context retrieval: the agent scans recent PRs and sees fetchWithRetry appears in 12 files added this month, always with the same error-handling wrapper.
3. Ranking adjustment: fetchWithRetry() moves to the top because merge frequency and recency both favor it.
4. Selection and testing: you accept the suggestion, write a test, commit.
5. Outcome observation: CI passes, the PR merges. The agent records a successful outcome for fetchWithRetry in this context.
6. Model update: next time anyone types fetch in a similar file, fetchWithRetry ranks even higher.
If the completion had caused a test failure or been revised during code review, the agent would demote that pattern for similar contexts. The loop is continuous—every commit, every test run, every review comment feeds back into the ranking model.
Balancing exploration and exploitation
A pure exploitation strategy—always ranking the most-merged pattern first—risks entrenching legacy code. If an old utility function has 500 historical uses but a new one is better, the autocomplete might never surface the new option.
Agent loops handle this with contextual exploration: when the developer is in a file recently modified, working on a feature branch that introduced a new pattern, or editing code written by someone who favors a specific style, the ranker boosts novelty. In stable parts of the codebase, it defaults to proven patterns. The exploration rate adapts to the change velocity of each directory.
Some systems also use explicit feedback prompts: if you reject the top suggestion three times in a row, the agent offers an alternative from lower in the ranked list and asks, "Is this closer?" The answer tunes the ranker for that context.
Practical implementation considerations
Data freshness: Repository signals go stale quickly. If your CI pipeline runs hourly, the ranking model should ingest results within minutes, not days. Batch updates work for merge frequency; test outcomes and review comments need near-real-time ingestion.
Privacy and scope: Not every signal should influence every completion. Code in a sensitive compliance module shouldn't leak patterns into a public-facing API client. Agent loops segment signals by directory, team ownership and access permissions—what you can't see doesn't train your autocomplete.
Fallback to static ranking: When repository history is sparse—new projects, unfamiliar languages, files with no recent activity—the agent falls back to language-model priors. Hybrid rankers combine repository signals with static heuristics, weighting repository data more heavily as it accumulates.
Performance: Querying Git history, test logs and review comments on every keystroke would be prohibitively slow. Practical systems precompute ranked candidate pools indexed by file path and symbol, refreshing them asynchronously whenever new code lands.
How Goatfied uses repository signals
Goatfied's AI code editor and cloud coding agents run a tight agent loop: when a cloud coding agent opens a pull request, it tests its own changes and observes CI outcomes. Those results—along with code review feedback and merge history—feed into the completion ranker. The next time you or a teammate work in a similar context, GOATs surface patterns that have already proven successful in your codebase, not just popular patterns from the open-source universe.
Because Goatfied integrates code, test runs, observability and team workflows on one platform, repository signals flow directly into the autocomplete engine without stitching together external tools. When an Operations GOAT deploys a change and observability telemetry shows an error spike, that signal can demote the offending code pattern in future completions—closing the loop from production back to the editor.
Frequently asked questions
How does agent-loop ranking differ from GitHub Copilot or similar tools?
Traditional AI code assistants rank completions primarily by language model probability and public code frequency. Agent-loop ranking adds feedback from your specific repository: merge history, test outcomes, code review comments and runtime behavior. The model learns your team's conventions, not just general programming patterns.
Can agent-loop ranking introduce bias toward legacy code?
Yes, if not designed carefully. A well-implemented agent loop uses recency weighting and contextual exploration to surface newer patterns when the developer is working in recently changed files or on feature branches. It balances proven patterns (exploitation) with emerging conventions (exploration).
What happens when there's not enough repository history to rank completions?
The system falls back to static ranking based on language model confidence and lexical scope. As your repository grows—more commits, PRs, test runs—repository signals gradually take precedence. Hybrid rankers blend both signals, increasing the weight on repository data as it becomes richer.
Does agent-loop ranking slow down autocomplete?
Not if implemented with precomputed indices. Rather than querying Git and CI logs on every keystroke, the system builds ranked candidate pools asynchronously and serves them from a fast in-memory cache. Index refreshes happen in the background whenever new code lands.
How do agent loops handle sensitive or private code?
Repository signals are scoped by directory permissions and team ownership. If you don't have access to a compliance module, its code patterns don't influence your completions. The agent only mines history from files and PRs you're authorized to see, respecting the same permission model as the rest of the platform.