Strong Build System
Codex reaches Level 3 with 75% Build System pass rate. Currently becoming autonomous-capable with 30/57 criteria passing (53%). Key areas for improvement include the opportunities listed below.
Strengths
01
Formatter
Prettier configured (.prettierrc.toml) for TypeScript, rustfmt configured (rustfmt.toml) for Rust
02
Lint Config
ESLint configured for TypeScript SDK/MCP (eslint.config.js), Clippy configured for Rust (clippy.toml with custom rules)
03
Naming Consistency
Clippy has disallowed-methods for naming conventions, ESLint has @typescript-eslint rules for naming
Opportunities
01
Cyclomatic Complexity
Add complexity analysis to identify and refactor overly complex functions.
02
Feature Flag Infrastructure
Add feature flags to enable safer deployments and gradual rollouts.
03
Integration Tests Exist
Add integration tests to verify component interactions and catch issues unit tests miss.
All Criteria
Style & Validation5/11 (45%)
—code_modularizationSkipped: Both applications are relatively small and focused. Rust's module system provides natural boundaries.
✗cyclomatic_complexityNo explicit cyclomatic complexity analysis configured in CI or linter configs
✗dead_code_detectionClippy detects unused code in Rust (enabled by default), but no dead code detection for TypeScript (no knip, ts-prune, or unimported)
✗duplicate_code_detectionNo duplicate code detection tools found (no jscpd, SonarQube, or similar)
✓formatterPrettier configured (.prettierrc.toml) for TypeScript, rustfmt configured (rustfmt.toml) for Rust
✗large_file_detectionNo large file detection tools found (no git hooks, CI checks, LFS config, or linter rules for file size)
✓lint_configESLint configured for TypeScript SDK/MCP (eslint.config.js), Clippy configured for Rust (clippy.toml with custom rules)
—n_plus_one_detectionSkipped: Applications do not have database/ORM usage requiring N+1 query detection
✓naming_consistencyClippy has disallowed-methods for naming conventions, ESLint has @typescript-eslint rules for naming
✗pre_commit_hooksNo pre-commit hooks found (no husky, lint-staged, or .pre-commit-config.yaml)
✓strict_typingTypeScript has strict mode enabled, Rust is strongly typed by default
✗tech_debt_trackingNo tech debt tracking found (no TODO/FIXME scanner in CI, no linter rules enforcing issue links, no SonarQube)
✓type_checkTypeScript has strict: true in tsconfig.json, Rust has compiler type checking by default
Build System9/12 (75%)
✗agentic_developmentNo evidence of agent co-authorship in recent 100 commits (no factory-droid, Claude, or other agent signatures)
—automated_pr_reviewSkipped: gh CLI is available but would require checking recent PRs for automated review comments, which is time-intensive for OSS evaluation
✓build_cmd_docREADME documents build commands: npm install -g @openai/codex or brew install --cask codex, AGENTS.md has just fmt and cargo commands
—build_performance_trackingSkipped: While sccache is used for build caching, no explicit build duration metrics or optimization tracking found
—dead_feature_flag_detectionSkipped: Prerequisite feature_flag_infrastructure failed (no feature flag system exists)
✓deployment_frequencyFrequent deployments: 5+ releases on Jan 22, 2026 alone (rust-v0.89.0-alpha.1 through alpha.5), multiple releases daily
✓deps_pinnedDependencies pinned: pnpm-lock.yaml and Cargo.lock are committed to repository
✓fast_ci_feedbackCI feedback is fast: most checks complete in 0-9 minutes (analyzed from recent merged PRs), well under 10-minute threshold
✗feature_flag_infrastructureNo feature flag infrastructure found (no LaunchDarkly, Statsig, Unleash, or GrowthBook configs). Statsig is only used for metrics via OpenTelemetry.
—heavy_dependency_detectionSkipped: CLI applications (non-bundled). Bundle size analysis not applicable to backend services or CLI tools.
✓monorepo_toolingMonorepo tooling configured: pnpm workspaces (pnpm-workspace.yaml) for TypeScript, Cargo workspace for Rust (codex-rs/Cargo.toml)
—progressive_rolloutSkipped: Not applicable for CLI application (not an infrastructure/service deployment)
✓release_automationRelease automation via rust-release.yml workflow: automated builds, signing, GitHub release creation, and npm publishing on tag push
✓release_notes_automationgit-cliff is configured (cliff.toml) and used in rust-release.yml workflow for automated changelog generation
—rollback_automationSkipped: Not applicable for CLI application (not an infrastructure/service deployment)
✓single_command_setupREADME documents single command setup: npm install -g @openai/codex followed by codex, or brew install --cask codex
✗unused_dependencies_detectioncargo-shear configured in CI (rust-ci.yml) for Rust dependencies, but no depcheck or similar for TypeScript packages
✓vcs_cli_toolsGitHub CLI (gh) is installed and authenticated successfully (verified with gh auth status)
—version_drift_detectionSkipped: No version drift detection tools found (no syncpack or manypkg for monorepo). Not critical for this monorepo structure.
Testing3/6 (50%)
—flaky_test_detectionSkipped: No evidence of flaky test management (no test retry configs, no BuildPulse, no quarantine mechanisms)
✗integration_tests_existRust has integration tests (login_tests, chatgpt_tests, app-server tests), but no integration tests found for TypeScript apps
✗test_coverage_thresholdsNo coverage thresholds enforced: jest.config.cjs has no coverageThreshold, no pytest --cov-fail-under, no CI coverage gates
✓test_isolationTests run in isolation: Jest runs parallel by default, Rust cargo test and nextest run tests in parallel
✓test_naming_conventionsTest naming enforced: Jest testMatch pattern for TypeScript (**/tests/**/*.test.ts), Rust follows *_test.rs and tests/ conventions
✗test_performance_trackingNo test performance tracking found (no timing output in CI, no test analytics platforms, no --durations flags in configs)
✓unit_tests_existUnit tests exist: TypeScript SDK/MCP have tests/**/*.test.ts files, Rust has extensive tests in codex-rs/**/tests/
—unit_tests_runnableSkipped: OSS evaluation mode - requires fully configured dev environment to verify test execution
Documentation3/7 (43%)
✓agents_mdAGENTS.md exists at repository root with Rust/codex-rs instructions, formatting rules, test commands, and TUI conventions
✗agents_md_validationNo AGENTS.md validation in CI (no automated checks that commands still work, no doc testing, no link checking)
—api_schema_docsSkipped: No API schema files found (no OpenAPI/Swagger/GraphQL schemas). Applications are CLI tools, not API services.
✗automated_doc_generationNo automated documentation generation found (no API doc generators, no changelog automation visible in workflows)
✓documentation_freshnessDocumentation is fresh: AGENTS.md was modified within last 180 days (git log shows recent updates)
✓readmeREADME.md exists with installation instructions, quickstart guide, and documentation links
✗service_flow_documentedNo architecture documentation found (no .mermaid, .puml files, no docs/architecture or docs/diagrams directories)
✗skillsNo skills directory found (.factory/skills/, .skills/, or .claude/skills/ do not exist)
Dev Environment1/2 (50%)
—database_schemaSkipped: Applications do not use databases (no Prisma schema, TypeORM entities, SQLAlchemy models, or SQL schemas)
✓devcontainer.devcontainer/devcontainer.json exists with Rust configuration, rust-analyzer and TOML extensions
—devcontainer_runnableSkipped: devcontainer CLI not installed, cannot verify container builds and runs successfully
✗env_templateNo environment template found (no .env.example, .env.template, or env.example file)
—local_services_setupSkipped: No external service dependencies requiring docker-compose.yml or local setup documentation
Debugging & Observability2/7 (29%)
✗alerting_configuredNo alerting infrastructure found (no PagerDuty, OpsGenie, or custom alerting rules in code or configs)
—circuit_breakersSkipped: Applications do not have external service dependencies requiring circuit breaker patterns
—code_quality_metricsSkipped: OSS evaluation mode - requires admin API access to check code scanning analyses and coverage bots
✗deployment_observabilityNo deployment observability documentation (no dashboard links in docs, no deploy notification integrations evident)
✓distributed_tracingOpenTelemetry configured for distributed tracing (codex-rs/otel/ with trace_exporter, X-Request-ID propagation evident)
✗error_tracking_contextualizedSentry configured in Rust (codex-feedback crate uses sentry crate 0.46), but no error tracking for TypeScript packages
—health_checksSkipped: CLI applications do not require health check endpoints (not deployed services with load balancers)
✓metrics_collectionMetrics collection via OpenTelemetry configured with Statsig as metrics exporter (STATSIG_OTLP_HTTP_ENDPOINT in otel config)
—profiling_instrumentationSkipped: Performance profiling not applicable for CLI applications (no APM tools, continuous profiling, or flame graph generation)
✗runbooks_documentedNo runbooks documentation found (no references to Notion, Confluence, wiki, or runbooks/ directory in README/AGENTS.md/docs)
✗structured_loggingRust uses tracing crate for structured logging (configured in multiple crates), but TypeScript packages have no logging library
Security4/6 (67%)
—automated_security_reviewSkipped: OSS evaluation mode - requires admin API access to check code-scanning/alerts for SAST tools
✓branch_protectionBranch protection via rulesets: 'alpha', 'CI must pass to merge to main', and 'Rust Release' rulesets exist (gh api rulesets)
✗codeownersNo CODEOWNERS file found in repository root or .github/ directory
—dast_scanningSkipped: Applications are not web services requiring DAST scanning (CLI tools, not deployed web applications)
✓dependency_update_automationDependabot configured (.github/dependabot.yaml) for cargo, github-actions, devcontainers, docker, rust-toolchain, and bun
✓gitignore_comprehensive.gitignore properly excludes .env* (not .env.example), node_modules, build artifacts, IDE configs, and OS files (.DS_Store)
✗log_scrubbingRust tracing crate has filtering/redaction capabilities configured, but TypeScript packages lack log sanitization
—pii_handlingSkipped: Applications do not process personal data requiring PII detection/handling (developer tools, no user data collection)
—privacy_complianceSkipped: CLI application without end-user data collection (no consent management, no GDPR/CCPA handling needed)
—secret_scanningSkipped: OSS evaluation mode - gh api secret-scanning/alerts returned 404 (not enabled or access denied)
✓secrets_management.env* files properly gitignored (not .env.example), GitHub Actions uses secrets.* references, no hardcoded secrets found
Task Discovery3/4 (75%)
✓backlog_healthExcellent backlog health: 100% of 50 open issues have labels, 0 issues older than 365 days
✗issue_labeling_systemLimited labeling system: only 'bug' label found from standard categories, missing priority (P0-P3), type (feature/chore), and area labels
✓issue_templates.github/ISSUE_TEMPLATE/ directory exists with structured templates (bug-report, feature-request, docs-issue, vs-code-extension)
✓pr_templates.github/pull_request_template.md exists with sections for description and testing
Product & Analytics0/2 (0%)
✗error_to_insight_pipelineNo error-to-insight pipeline found (no Sentry-GitHub integration, no automated issue creation from errors)
✗product_analytics_instrumentationStatsig metrics instrumentation via OpenTelemetry (used for telemetry), but no full product analytics like Mixpanel/Amplitude