Factory.ai

Strong Build System

pi-mono reaches Level 3 with 67% Build System pass rate. Currently becoming autonomous-capable with 24/57 criteria passing (42%). Key areas for improvement include the opportunities listed below.

Strengths

01
Formatter
biome.json configures formatting (indentStyle, indentWidth, lineWidth) for all applications.
02
Lint Config
biome.json configures linting for all TypeScript packages with recommended rules enabled.
03
Pre Commit Hooks
.husky/pre-commit runs 'npm run check' (formatting, linting, type checking) for all packages.

Opportunities

01
Cyclomatic Complexity
Add complexity analysis to identify and refactor overly complex functions.
02
Feature Flag Infrastructure
Add feature flags to enable safer deployments and gradual rollouts.
03
Monorepo Tooling
Consider Turborepo or Nx for better caching, incremental builds, and dependency graph awareness.

All Criteria

Style & Validation5/11 (45%)
—code_modularizationSkipped: no module boundary enforcement tools (eslint-plugin-boundaries, dependency-cruiser, import-linter) configured.
✗cyclomatic_complexityNo complexity analysis tools configured (ESLint complexity rule, SonarQube, or other analyzers).
✗dead_code_detectionNo dead code detection tools configured (knip, ts-prune, unimported, ESLint no-unused-vars strict mode).
✗duplicate_code_detectionNo duplicate code detection tools configured (jscpd, PMD CPD, SonarQube).
✓formatterbiome.json configures formatting (indentStyle, indentWidth, lineWidth) for all applications.
✗large_file_detectionNo file size detection tooling found: no git hooks checking file size, no CI jobs flagging large files, no .gitattributes with LFS, no linter rules for file size.
✓lint_configbiome.json configures linting for all TypeScript packages with recommended rules enabled.
—n_plus_one_detectionSkipped: CLI tools without database/ORM usage, not applicable.
✗naming_consistencyNo naming convention enforcement found: no @typescript-eslint/naming-convention rules, no documented naming standards in AGENTS.md.
✓pre_commit_hooks.husky/pre-commit runs 'npm run check' (formatting, linting, type checking) for all packages.
✓strict_typingtsconfig.base.json enables strict TypeScript mode for all applications.
✗tech_debt_trackingNo TODO/FIXME scanner in CI, no SonarQube/SonarCloud configuration, no documented tech debt tracking system.
✓type_checktsconfig.base.json has 'strict': true for all TypeScript applications (coding-agent, mom, pods).
Build System8/12 (67%)
✗agentic_developmentNo evidence of agent co-authorship in git log, no agent CI workflows, no .factory/droids/ or agent configuration directories.
—automated_pr_reviewSkipped: OSS evaluation mode without admin API access. PR gate workflow exists but doesn't generate review comments.
✓build_cmd_docREADME.md documents 'npm run build' command clearly in Development section.
—build_performance_trackingSkipped: no evidence of build caching metrics, no build metrics exported to monitoring, would fail in OSS mode.
—dead_feature_flag_detectionSkipped: prerequisite feature_flag_infrastructure not configured.
✓deployment_frequencyVery frequent releases: 30 releases in ~15 days (multiple per day/week). gh release list shows active deployment culture.
✓deps_pinnedpackage-lock.json exists and is committed, ensuring dependency versions are pinned.
—fast_ci_feedbackSkipped: requires authenticated gh CLI access to PR statusCheckRollup data not available for OSS evaluation.
✗feature_flag_infrastructureNo feature flag system configured (LaunchDarkly, Statsig, Unleash, GrowthBook, or custom system).
—heavy_dependency_detectionSkipped: no bundle analysis configured (webpack-bundle-analyzer, size-limit), not critical for CLI tools.
✗monorepo_toolingUses npm workspaces only without sophisticated monorepo tooling (Turborepo, Nx, Lerna) for incremental builds and caching.
—progressive_rolloutSkipped: CLI tools and library repository, not an infrastructure deployment with canary/percentage rollouts.
✓release_automationAutomated release pipeline: scripts/release.mjs handles version bumping, changelog, commit, tag, publish; build-binaries.yml handles binary builds.
✓release_notes_automationscripts/release.mjs automatically updates CHANGELOG.md files when releasing: [Unreleased] → [version] with date.
—rollback_automationSkipped: CLI tools and library repository, not an infrastructure deployment requiring rollback automation.
✓single_command_setupREADME documents simple setup: 'npm install && npm run dev' gets you from fresh clone to running dev environment.
✗unused_dependencies_detectionNo unused dependency detection tools configured (depcheck, npm-check, knip).
✓vcs_cli_toolsGitHub CLI (gh) is installed and authenticated (verified with gh auth status).
✓version_drift_detectionscripts/sync-versions.js keeps package versions in sync across monorepo (used in version:patch/minor/major commands).
Testing2/7 (29%)
—flaky_test_detectionSkipped: no test retry configuration, no flaky test tracking tools, no PR check analysis available.
✗integration_tests_existNo integration test directories (cypress/, playwright, integration/) found in any of the 3 applications.
✗test_coverage_thresholdsNo coverage thresholds: vitest.config.ts missing coverageThreshold, no Codecov/Coveralls PR checks in CI.
✓test_isolationvitest runs tests in parallel by default, providing test isolation for all packages with tests.
✓test_naming_conventionsvitest implicit *.test.ts naming convention enforced. All test files follow this pattern.
✗test_performance_trackingvitest.config.ts has testTimeout but no test timing output, no test analytics platforms, no duration tracking.
✗unit_tests_existOnly coding-agent has test/ directory with unit tests. mom and pods have no test scripts or test directories.
✗unit_tests_runnablecoding-agent tests verified runnable (vitest --run executed successfully). mom and pods have no tests.
Documentation4/7 (57%)
✓agents_mdAGENTS.md exists at repo root with comprehensive development rules, code quality guidelines, commands, and workflow instructions.
✗agents_md_validationNo CI validation of AGENTS.md commands, no automated AGENTS.md generation/update, no documentation testing.
—api_schema_docsSkipped: CLI tools without HTTP APIs, no OpenAPI/Swagger/GraphQL schemas expected.
✓automated_doc_generationscripts/release.mjs automatically generates/updates CHANGELOG.md documentation during releases (automated changelog generation).
✓documentation_freshnessAGENTS.md was modified within last 180 days (git log verification passed).
✓readmeREADME.md exists at repo root with package descriptions, development instructions, and contribution guidelines.
✗service_flow_documentedNo architecture diagrams (.mermaid, .puml files) or service dependency documentation found in repository.
✗skillsNo skills directories found (.factory/skills/, .skills/, .claude/skills/) in repository.
Dev Environment0/2 (0%)
—database_schemaSkipped: no database schema files (Prisma, TypeORM) found, CLI tools don't use databases.
✗devcontainerNo .devcontainer/devcontainer.json configuration found.
—devcontainer_runnableSkipped: no devcontainer configuration exists to test.
✗env_templateNo .env.example file found, environment variables not comprehensively documented in README or AGENTS.md.
—local_services_setupSkipped: no docker-compose.yml found, applications (CLI tools) do not require external service dependencies.
Debugging & Observability0/7 (0%)
✗alerting_configuredNo alerting services (PagerDuty, OpsGenie) or custom alerting rules configured.
—circuit_breakersSkipped: no circuit breaker libraries or patterns found, not critical for CLI tools.
—code_quality_metricsSkipped: requires admin API access to check code-scanning analyses, not available for OSS evaluation.
✗deployment_observabilityNo references to monitoring dashboards or deployment notification integrations in documentation.
✗distributed_tracingSome trace ID references in code but no OpenTelemetry or distributed tracing implementation configured.
✗error_tracking_contextualizedNo error tracking services (Sentry, Bugsnag, Rollbar) configured in any application.
—health_checksSkipped: CLI tools not deployed as services, health checks not applicable.
✗metrics_collectionNo metrics/telemetry instrumentation (Datadog, Axiom, Prometheus, New Relic, CloudWatch) configured.
—profiling_instrumentationSkipped: no profiling tools (APM, Pyroscope, clinic.js) configured.
✗runbooks_documentedNo references to runbooks/playbooks found in README, AGENTS.md, or docs/ directories.
✗structured_loggingNo structured logging libraries (winston, pino, bunyan) or custom logger modules found in dependencies.
Security3/6 (50%)
—automated_security_reviewSkipped: requires admin API access to check code-scanning alerts, not available for OSS evaluation.
✓branch_protectionActive ruleset found via gh API: 'Copilot review for default branch' ruleset enforces protection on main branch.
✗codeownersNo CODEOWNERS file found in repository root or .github/ directory.
—dast_scanningSkipped: CLI tools not deployed as web services, DAST not applicable.
✗dependency_update_automationNo Dependabot or Renovate configuration files found (.github/dependabot.yml, renovate.json).
✓gitignore_comprehensive.gitignore properly excludes .env (not .env.example), node_modules, dist, .DS_Store, .idea, .vscode, and other sensitive files.
✗log_scrubbingNo log sanitization/scrubbing mechanisms configured (no redaction support in logging configuration).
—pii_handlingSkipped: dev tools without personal data processing, not applicable.
—privacy_complianceSkipped: CLI tools and dev utilities without end-user data collection, not applicable.
—secret_scanningSkipped: requires admin API access not available for OSS evaluation.
✓secrets_managementGitHub Actions uses secrets.* references properly, .env files gitignored, no hardcoded secrets found.
Task Discovery2/3 (67%)
—backlog_healthSkipped: recent issues lack labels (< 1 day old), cannot determine if >70% meet criteria without full analysis.
✓issue_labeling_systemConsistent labeling system: pkg:* labels (ai, coding-agent, mom, pods, tui, web-ui), bug/enhancement type labels.
✓issue_templates.github/ISSUE_TEMPLATE/ exists with bug.yml, contribution.yml, config.yml structured templates.
✗pr_templatesNo .github/pull_request_template.md or .github/PULL_REQUEST_TEMPLATE.md found.
Product & Analytics0/2 (0%)
✗error_to_insight_pipelineNo Sentry-GitHub integration or automated error-to-issue pipeline configured.
✗product_analytics_instrumentationNo product analytics (Mixpanel, Amplitude, PostHog, Heap, GA4) instrumented in applications.

start building

Ready to build the software of the future?

Start building

Arrow Right Icon