Datadog Uses OpenAI Codex for System-Level Code Review

Background and Deployment Context

As a cloud observability platform, Datadog relies heavily on infrastructure-as-code (IaC) and systems programming languages (e.g., Go, Python) to build its highly available monitoring stack. To ensure deployment consistency and enforce security policies, it requires rigorous review of cross-team system-level changes.

Technical Implementation

• Leverages OpenAI Codex (not GPT-4 or later models) as the core code understanding and generation engine, with custom prompt engineering tailored to vertical tasks: Terraform configuration validation, Ansible playbook security constraints, and Go module dependency auditing;
• Functions as an automated gatekeeper—not a replacement for human code review—identifying common anti-patterns (e.g., unencrypted S3 buckets, hardcoded secrets, over-provisioned IAM permissions) and generating actionable remediation suggestions;
• Integrates directly into GitHub PR workflows and synchronizes with Datadog’s internal Policy-as-Code rule repository, enabling dynamic policy updates and feedback loops.

Measured Outcomes

• Achieves throughput of 120+ PRs reviewed per day, with median latency <8 seconds;
• In pilot phase, increased detection rate of infrastructure configuration defects by 37%, and reduced manual review time by 52%;
• All Codex invocations run within Datadog’s private VPC; no input or output leaves the enterprise boundary, meeting SOC2 Type II compliance requirements.