Factory.ai

Strong Testing

containerd reaches Level 4 with 86% Testing pass rate. Currently reaching production grade with 39/60 criteria passing (65%). Key areas for improvement include the opportunities listed below.

Strengths

01
Testing (86%)
Includes Integration Tests Exist, Test Coverage Thresholds, Test Isolation.
02
Code Modularization
✓ internal/ package directories enforce Go's compiler-enforced visibility boundaries
03
Dead Code Detection
✓ staticcheck and revive detect unused code; unused-parameter checks in golangci config

Opportunities

01
Cyclomatic Complexity
Add complexity analysis to identify and refactor overly complex functions.
02
Feature Flag Infrastructure
Add feature flags to enable safer deployments and gradual rollouts.
03
Skills
Create .factory/skills to give agents reusable, tested capabilities for common tasks.

All Criteria

Style & Validation8/12 (67%)
✓code_modularization✓ internal/ package directories enforce Go's compiler-enforced visibility boundaries
✗cyclomatic_complexity✗ No gocyclo or complexity rules found in golangci config; no complexity thresholds enforced
✓dead_code_detection✓ staticcheck and revive detect unused code; unused-parameter checks in golangci config
✓duplicate_code_detection✓ dupword linter checks for duplicate words in source code
✓formatter✓ gofmt and goimports configured in .golangci.yml formatters section
✗large_file_detection✗ No git hooks, CI file size checks, or .gitattributes LFS configuration found
✓lint_config✓ .golangci.yml configured with gosec, revive, misspell, depguard, and other linters
—n_plus_one_detectionSkipped - infrastructure library without visible ORM/database query patterns
✓naming_consistency✓ revive linter with package-comments and naming convention rules enabled
✗pre_commit_hooks✗ No .pre-commit-config.yaml or husky found; no git hooks configured
✓strict_typing✓ Go compiler enforces strict typing by default; no dynamic typing escape hatches used
✗tech_debt_tracking✗ No TODO/FIXME scanner in CI; no linter rules enforcing tech debt tracking
✓type_check✓ Go is strongly typed by default; go.mod requires go >= 1.24.3
Build System8/11 (73%)
✗agentic_development✗ No agent co-authorship found in recent 100 commits; no droid/factory/claude signatures
—automated_pr_reviewSkipped - would require analyzing PR comments for bot/automation; insufficient clear evidence
✓build_cmd_doc✓ BUILDING.md and README.md document 'make' and 'go build' commands clearly
✗build_performance_tracking✗ No build caching (turbo/nx), metrics export, or deliberate build optimization tracking found
—dead_feature_flag_detectionSkipped - prerequisite fails (no feature flag infrastructure)
✓deployment_frequency✓ High release cadence: 2.2.1 (Dec 18), 2.1.6 (Dec 18), 1.7.30 (Dec 18) - multiple per month
✓deps_pinned✓ go.sum committed; Go modules provide cryptographic dependency locking
—fast_ci_feedbackSkipped - would require calculating CI duration from statusCheckRollup timestamps
✗feature_flag_infrastructure✗ No LaunchDarkly, Statsig, Unleash, or custom feature flag system configured
—heavy_dependency_detectionSkipped - not a bundled/frontend application; no bundle size analysis needed
—monorepo_toolingSkipped - single-application repository; no monorepo tooling needed
—progressive_rolloutSkipped - infrastructure library; progressive rollout managed by downstream users/operators
✓release_automation✓ .github/workflows/release.yml automates binary builds and releases on tag push
✓release_notes_automation✓ release.yml extracts release notes from git tag annotations automatically
—rollback_automationSkipped - infrastructure library; rollback handled by package managers and orchestrators
✓single_command_setup✓ BUILDING.md documents 'git clone && cd containerd && make' for dev setup
✓unused_dependencies_detection✓ CI runs 'make verify-vendor' which includes go mod tidy to detect unused dependencies
✓vcs_cli_tools✓ gh CLI available and authenticated to github.com (jonathan-factory account)
—version_drift_detectionSkipped - single-application repository; version drift not applicable
Testing6/7 (86%)
—flaky_test_detectionSkipped - would require analyzing PR statusCheckRollup for retry patterns; no clear evidence
✓integration_tests_exist✓ integration/ directory with comprehensive integration tests; cri-integration tests
✓test_coverage_thresholds✓ codecov.yml exists; Makefile has coverage, root-coverage, cri-integration-coverage targets
✓test_isolation✓ Tests run with -parallel flag in Makefile; TESTFLAGS_PARALLEL variable configured
✓test_naming_conventions✓ Go enforces *_test.go naming convention by default; test files follow this pattern
✓test_performance_tracking✓ Makefile has coverage targets; CI workflows output test results and timing information
✓unit_tests_exist✓ Extensive *_test.go files throughout codebase (cmd/, core/, pkg/, internal/)
—unit_tests_runnableRequires local test execution environment to verify (skipped).
Documentation5/7 (71%)
✗agents_md✗ AGENTS.md not found in repository root
—agents_md_validationSkipped - prerequisite fails (AGENTS.md not found)
✓api_schema_docs✓ Protobuf schemas in api/ directory define gRPC APIs (events.proto, version.proto, etc.)
✓automated_doc_generation✓ Makefile has 'man' target generating man pages from markdown using go-md2man
✓documentation_freshness✓ CONTRIBUTING.md updated in last 180 days (recent activity)
✓readme✓ Comprehensive README.md with project overview, installation, and usage instructions
✓service_flow_documented✓ Architecture diagrams exist: docs/historical/design/architecture.png, data-flow.png, cri/architecture.png
✗skills✗ No .factory/skills/, .skills/, or .claude/skills/ directories found
Dev Environment1/2 (50%)
—database_schemaSkipped - infrastructure daemon without traditional database schema; uses bbolt for metadata
✓devcontainer✓ .devcontainer/devcontainer.json configured with Go 1.24.11 and docker-in-docker
—devcontainer_runnableSkipped - devcontainer CLI availability not verified
✗env_template✗ No .env.example file; environment variables not documented in README/AGENTS.md
—local_services_setupSkipped - infrastructure daemon; no external services (postgres/redis) required for local dev
Debugging & Observability5/8 (63%)
—alerting_configuredSkipped - infrastructure library/daemon; alerting typically configured by users/operators
—circuit_breakersSkipped - infrastructure daemon; circuit breakers not typical for this type of system
✓code_quality_metrics✓ codecov.yml configured for coverage tracking; CI generates coverage profiles
✗deployment_observability✗ No monitoring dashboard links (Datadog, Grafana) found in documentation
✓distributed_tracing✓ OpenTelemetry instrumentation: go.opentelemetry.io/otel packages with grpc/http instrumentation
✗error_tracking_contextualized✗ No Sentry, Bugsnag, or Rollbar integration found for production error tracking
—health_checksSkipped - daemon/runtime provides health via gRPC API; not a traditionally deployed web service
✓metrics_collection✓ Prometheus client instrumented: github.com/prometheus/client_golang in dependencies
✓profiling_instrumentation✓ internal/pprof directory exists for Go profiling support
✗runbooks_documented✗ No runbooks or links to incident response procedures found in README/docs
✓structured_logging✓ github.com/sirupsen/logrus and github.com/containerd/log in go.mod for structured logging
Security3/8 (38%)
✓automated_security_review✓ CodeQL SAST configured in .github/workflows/codeql.yml; runs on PRs and pushes
✓branch_protection✓ Active rulesets: tag create/delete protections enforced via GitHub rulesets
✗codeowners✗ No CODEOWNERS file in repository root or .github/ directory
—dast_scanningSkipped - not a web application; DAST not applicable to container runtime daemon
✓dependency_update_automation✓ Dependabot configured for go modules and github-actions; weekly updates with grouping
✗gitignore_comprehensive✗ .gitignore missing common patterns: no .env, .idea, .vscode, .DS_Store entries
✗log_scrubbing✗ No log sanitization/redaction mechanism configured in logging setup
—pii_handlingSkipped - infrastructure daemon without user PII processing
—privacy_complianceSkipped - infrastructure daemon without end-user data collection
—secret_scanningRequires GitHub admin API access to verify secret scanning configuration (skipped).
✗secrets_management✗ No cloud secrets manager integration or secrets management pattern evident
Task Discovery3/4 (75%)
✓backlog_health✓ 49/50 open issues analyzed have descriptive titles (>10 chars) AND labels (98% health)
✓issue_labeling_system✓ Comprehensive labels: area/* (cri,distribution,runtime), kind/* (bug,feature), priority/P1, platform/windows
✓issue_templates✓ .github/ISSUE_TEMPLATE/ with bug_report.yaml, feature_request.yaml, cri_kep.yaml
✗pr_templates✗ No .github/pull_request_template.md or PULL_REQUEST_TEMPLATE.md found
Product & Analytics0/1 (0%)
✗error_to_insight_pipeline✗ No Sentry-GitHub integration or automated error-to-issue pipeline found
—product_analytics_instrumentationSkipped - infrastructure project; no end-user product analytics needed

start building

Ready to build the software of the future?

Start building

Arrow Right Icon