Running AI coding assistants in air-gapped environments

Air-gapped environments—networks physically isolated from the internet and other external systems—pose a unique challenge for AI coding assistants. Most commercial AI tools rely on cloud APIs, making them incompatible with the security perimeter required by defense contractors, financial institutions, and organizations handling classified data. The core problem: you need the productivity gains of AI-assisted development while maintaining complete network isolation.

This article walks through the technical requirements, architecture patterns, and operational tradeoffs for running AI coding assistants behind an air gap, with practical guidance for security teams and engineering leaders evaluating deployment options.

Understanding air-gapped deployment requirements

An air-gapped environment has no physical or logical connection to external networks. Data enters and exits only through controlled media transfer—typically removable drives scanned at a transfer station. For AI coding assistants, this means:

• No cloud API calls: Services like GitHub Copilot, Cursor's default mode, and ChatGPT integration cannot function
• Local model inference: All language models must run on infrastructure within the secure perimeter
• Offline update mechanisms: Model updates, security patches, and feature releases require manual transfer procedures
• Self-contained training pipelines: If you fine-tune models on internal code, the entire MLOps stack must exist inside the air gap

The regulatory drivers vary by industry. Defense contractors working on classified programs face NISPOM and ITAR requirements. Financial institutions may operate air-gapped trading systems under operational resilience rules. Healthcare organizations handling research data contend with HIPAA's physical safeguard provisions.

Architecture patterns for isolated AI coding tools

Three deployment patterns dominate the air-gapped AI coding assistant landscape:

On-premises model serving

Deploy open-weight models (Llama, CodeLlama, StarCoder, DeepSeek Coder) on GPU infrastructure within your secure facility. Developers connect to an internal API endpoint that mimics OpenAI's interface. This approach gives you:

• Full control over model selection and updates
• Ability to fine-tune on proprietary codebases
• Integration with existing IDE extensions via localhost or internal domains

The infrastructure cost is substantial: expect 4-8 A100 or H100 GPUs for acceptable latency on a 34B parameter model serving a team of 50 engineers. Smaller models (7B-13B parameters) run on fewer GPUs but produce less sophisticated code suggestions.

Bundled desktop applications

Some vendors package model weights and inference engines into desktop applications that run entirely on developer workstations. Each machine performs local inference using CPU or integrated GPU. This eliminates central infrastructure but:

• Requires beefy developer hardware (32GB+ RAM, modern multi-core processors)
• Limits model size to what a single workstation can run (typically 7B parameters or less)
• Complicates model updates since each machine needs the new weights transferred individually

Hybrid data diode systems

Organizations with extremely sensitive requirements sometimes use unidirectional network connections (data diodes) to allow model inference requests out while blocking any data return path that could exfiltrate code. This is technically not air-gapped, but it addresses the exfiltration risk while maintaining some connectivity. Implementation complexity is high and trust assumptions must be carefully documented.

Operational tradeoffs and hidden costs

Running AI coding assistants in air-gapped environments introduces operational overhead that cloud-connected alternatives don't face:

Model staleness: Commercial cloud services continuously improve their models. An air-gapped deployment falls behind unless your team establishes a regular model update cadence—quarterly at minimum. Each update requires security review, transfer media preparation, and deployment validation.

Limited training data: Fine-tuning models on your internal codebase to improve relevance requires maintaining MLOps infrastructure (data pipelines, training clusters, evaluation harnesses) entirely within the air gap. Many organizations lack this capability and must accept generic model behavior.

Integration constraints: Popular coding assistant features like repository-wide semantic search, PR review automation, and CI/CD integration require additional isolated infrastructure. You're not just deploying a model—you're replicating an entire development toolchain behind the perimeter.

Audit and compliance burden: Every component that crosses the air gap needs security review. Model weights, inference code, IDE extensions, and dependency updates all require formal approval and scanning. This slows the adoption of new capabilities compared to automatically-updated cloud tools.

Evaluating vendors for secure environments

When assessing AI coding assistant vendors for air-gapped deployment, focus on these technical and contractual points:

• Licensing for offline use: Confirm the license explicitly permits deployment without internet connectivity and clarify whether per-seat pricing applies inside the air gap
• Model provenance: Request documentation of training data sources, especially whether the model was trained on public code repositories (which may create IP concerns)
• Update delivery mechanism: Understand how the vendor delivers model updates, security patches, and new features—USB drives, secure FTP to a transfer station, or other methods
• Support model: Determine whether support staff can access logs and diagnostics, or if you're limited to email-only support with manual log export
• Telemetry and phoning home: Verify the software contains no analytics, crash reporting, or license verification that assumes internet connectivity

It's worth noting that platforms like Goatfied are not designed for air-gapped deployment and require internet connectivity for their cloud-based AI agents and coding assistants. Organizations with air-gap requirements need purpose-built solutions or self-managed open-source stacks.

Implementation checklist for security teams

If you're tasked with deploying an AI coding assistant in an air-gapped environment, this checklist covers the critical workstreams:

Infrastructure preparation

• [ ] Provision GPU servers or approve desktop hardware specs for local inference
• [ ] Establish internal API endpoint or distribution mechanism for desktop apps
• [ ] Configure monitoring for inference latency, throughput, and GPU utilization
• [ ] Set up model registry and version control for weights inside the perimeter

Security review

• [ ] Audit model training data provenance and licensing
• [ ] Review inference code for any external API calls or telemetry
• [ ] Verify IDE extensions make no network requests outside the air gap
• [ ] Document data flows in your security architecture diagrams

Operational procedures

• [ ] Define model update cadence and approval workflow
• [ ] Create transfer media handling procedures (scanning, chain of custody)
• [ ] Train developers on tool usage and limitations vs. cloud alternatives
• [ ] Establish metrics for measuring productivity impact and adoption

Compliance documentation

• [ ] Update system security plans to include AI coding assistant components
• [ ] Document risk acceptance for model staleness vs. cloud-connected tools
• [ ] Prepare accreditation materials if required by your regulatory framework

Frequently asked questions

Can I use GitHub Copilot in an air-gapped environment?

GitHub Copilot requires continuous internet connectivity to GitHub's cloud API and cannot function in a true air-gapped environment. Microsoft offers GitHub Copilot for Business with enhanced data protection, but not an on-premises deployment option. Organizations needing air-gapped AI coding assistance must evaluate alternatives like self-hosted open-weight models or vendors offering isolated deployment modes.

What model size do I need for acceptable code completion quality?

For general-purpose code completion and explanation, 7B parameter models provide basic functionality but often produce less contextually appropriate suggestions than larger models. Most teams find 13B-34B parameter models offer a better tradeoff between infrastructure cost and suggestion quality. Models above 70B parameters deliver diminishing returns for the GPU investment unless you're also doing complex code generation or architecture discussions with the AI.

How often should I update models in an air-gapped deployment?

Quarterly updates represent a reasonable minimum cadence for most organizations. This balances the operational overhead of transfer, review, and deployment against model improvements. More frequent updates make sense if you're fine-tuning on internal code that changes rapidly or if your developers report specific quality issues the vendor has addressed. Less frequent updates (semi-annual or annual) significantly increase the capability gap between your tools and cloud-connected alternatives.

Do air-gapped AI coding assistants pose insider threat risks?

Yes—the model runs inside your perimeter with access to source code, creating a potential exfiltration vector if a malicious insider modifies the inference server or desktop application. Mitigation strategies include: treating inference infrastructure as high-trust systems with enhanced access controls, monitoring for unusual API access patterns, and maintaining the same code review and commit approval processes you use for other sensitive changes. The model itself doesn't inherently create risk, but the infrastructure around it requires security hardening.

Can I train custom models on my company's codebase in an air-gapped environment?

Technically yes, but most organizations underestimate the infrastructure and expertise required. You need: GPU clusters for training (larger than inference clusters), data pipeline tooling to prepare code samples, evaluation frameworks to measure quality, and ML engineers to tune hyperparameters and iterate on model architecture. Unless you're already operating ML workloads inside the air gap, the cost and timeline typically exceed the value for coding assistance alone. Starting with a well-performing open-weight model and fine-tuning only if generic performance proves insufficient is the pragmatic path.

Related reading

• Self-hosted AI coding assistants: a complete Goatfied deployment guide