Factory.ai

Strong Build System

Assistant reaches Level 2 with 62% Build System pass rate. Currently building team readiness with 19/60 criteria passing (32%). Key areas for improvement include the opportunities listed below.

Strengths

01
Formatter
All 7 apps use Prettier formatter (.prettierrc at root)
02
Lint Config
All 7 apps use ESLint with TypeScript support (eslint.config.js at root)
03
Strict Typing
All 7 apps inherit strict: true from tsconfig.base.json

Opportunities

01
Cyclomatic Complexity
Add complexity analysis to identify and refactor overly complex functions.
02
Feature Flag Infrastructure
Add feature flags to enable safer deployments and gradual rollouts.
03
Integration Tests Exist
Add integration tests to verify component interactions and catch issues unit tests miss.

All Criteria

Style & Validation4/12 (33%)
✗code_modularizationNo module boundary enforcement (no eslint-plugin-boundaries, dependency-cruiser, or Nx boundaries)
✗cyclomatic_complexityNo complexity analysis tools (no ESLint complexity rule, lizard, radon, or SonarQube)
✗dead_code_detectionNo dead code detection tools (no knip, ts-prune, unimported, or similar)
✗duplicate_code_detectionNo duplicate code detection (no jscpd, PMD CPD, or SonarQube)
✓formatterAll 7 apps use Prettier formatter (.prettierrc at root)
✗large_file_detectionNo tooling found for large file detection (no git hooks, CI checks, or LFS configuration)
✓lint_configAll 7 apps use ESLint with TypeScript support (eslint.config.js at root)
—n_plus_one_detectionSkipped: Uses better-sqlite3 with no ORM, N+1 detection not applicable
✗naming_consistencyNo naming convention enforcement (no ESLint naming-convention rule or documented standards)
✗pre_commit_hooksNo pre-commit hooks configured (no Husky, lint-staged, or .pre-commit-config.yaml)
✓strict_typingAll 7 apps inherit strict: true from tsconfig.base.json
✗tech_debt_trackingNo tech debt tracking found (no TODO scanner, linter rules, or SonarQube configuration)
✓type_checkAll 7 apps use TypeScript with strict type checking enabled
Build System8/13 (62%)
✗agentic_developmentNo evidence of AI agent co-authorship in git history (checked last 100 commits)
—automated_pr_reviewSkipped: gh CLI available but no automated review comments found in recent PRs
✓build_cmd_docREADME.md documents build commands: 'npm install', 'npm run build', 'npm run start'
—build_performance_trackingSkipped: No CI/CD to track build performance metrics
—dead_feature_flag_detectionSkipped: Prerequisite failed (no feature flag infrastructure)
✓deployment_frequencyHigh release frequency: 14 releases in ~2 weeks (multiple releases per week via GitHub releases)
✓deps_pinnedpackage-lock.json is committed to repository, ensuring reproducible builds
—fast_ci_feedbackSkipped: No CI/CD configured (statusCheckRollup empty for all merged PRs)
✗feature_flag_infrastructureNo feature flag system found (no LaunchDarkly, Statsig, Unleash, or custom implementation)
✗heavy_dependency_detectionNo bundle size analysis (no webpack-bundle-analyzer, size-limit, or bundlesize configured)
✓monorepo_toolingnpm workspaces configured in package.json with 9 workspace packages
—progressive_rolloutSkipped: Not an infrastructure repository, progressive rollout not applicable
✓release_automationscripts/release.mjs automates version bumping, changelog updates, and GitHub release creation
✓release_notes_automationCHANGELOG.md maintained with automated release script (scripts/release.mjs)
—rollback_automationSkipped: Not an infrastructure repository, automated rollback not applicable
✓single_command_setupREADME documents simple setup: 'npm install && npm run build && npm run start'
✗unused_dependencies_detectionNo unused dependency detection (no depcheck, npm-check, or knip)
✓vcs_cli_toolsgh CLI is installed and authenticated (verified with 'gh auth status')
✗version_drift_detectionNo version drift detection (no syncpack, manypkg, or similar tooling in monorepo)
Testing2/6 (33%)
—flaky_test_detectionSkipped: No CI to check for test retries or flaky test patterns
✗integration_tests_existNo integration tests found (no Cypress, Playwright, or e2e test directories)
✗test_coverage_thresholdsNo coverage thresholds configured in vitest.config.ts (no coverageThreshold setting)
✗test_isolationNo test isolation enforcement (no parallel test configuration or database isolation patterns)
✓test_naming_conventionsAll 7 apps enforce test naming via Vitest include patterns (*.test.ts files)
✗test_performance_trackingNo test performance tracking (no explicit timing output or test analytics integration)
✓unit_tests_existAll 7 apps have extensive test coverage with *.test.ts files (100+ test files found)
—unit_tests_runnableSkipped: OSS evaluation mode - requires full dev environment setup not available
Documentation3/7 (43%)
✓agents_mdAGENTS.md exists at repository root with development guidelines
✗agents_md_validationNo automation to validate AGENTS.md accuracy (no CI checks or documentation testing)
—api_schema_docsSkipped: No OpenAPI/Swagger or GraphQL schema files found (not API-focused apps)
✗automated_doc_generationNo automated documentation generation (no API doc generators, JSDoc extraction, or doc automation)
✓documentation_freshnessAGENTS.md was modified in the last 180 days (verified with git log)
✓readmeREADME.md exists at repository root with comprehensive setup and usage instructions
✗service_flow_documentedNo architecture diagrams found (no .mermaid, .puml files, or docs/architecture directory)
✗skillsNo skills directories found (.factory/skills, .skills, or .claude/skills)
Dev Environment0/3 (0%)
✗database_schemabetter-sqlite3 used but no schema definition files (no Prisma, TypeORM, or SQL schema files)
✗devcontainerNo .devcontainer directory or devcontainer.json configuration
—devcontainer_runnableSkipped: Prerequisite failed (no devcontainer configured)
✗env_templateNo .env.example file, and environment variables not documented in README or AGENTS.md
—local_services_setupSkipped: No docker-compose.yml for local service dependencies
Debugging & Observability1/9 (11%)
✗alerting_configuredNo alerting configured (no PagerDuty, OpsGenie, or custom alerting rules)
✗circuit_breakersNo circuit breaker implementation (no opossum, cockatiel, or service mesh configuration)
—code_quality_metricsSkipped: OSS evaluation mode - requires admin API access to check code scanning
✗deployment_observabilityNo deployment observability (no monitoring dashboard links or deploy notification integrations)
✗distributed_tracingNo distributed tracing (no OpenTelemetry, X-Request-ID headers, or trace-id propagation)
✗error_tracking_contextualizedNo error tracking (no Sentry, Bugsnag, or Rollbar configuration)
✗health_checks2 of 7 apps have health check endpoints: coding-sidecar and notify-proxy implement /health
✗metrics_collectionNo metrics instrumentation (no Datadog, Prometheus, New Relic, or CloudWatch)
—profiling_instrumentationSkipped: Profiling not applicable for most apps (libraries, CLIs)
✓runbooks_documenteddocs/REPO_DOCUMENTATION_GUIDELINES.md references runbooks location (docs/operations/)
✗structured_loggingNo structured logging libraries (no winston, pino, bunyan, or custom logger modules)
Security1/5 (20%)
—automated_security_reviewSkipped: OSS evaluation mode - GitHub API returned 403 (insufficient permissions)
—branch_protectionSkipped: OSS evaluation mode - requires admin API access not available for public repos
✗codeownersNo CODEOWNERS file in root or .github/ directory
—dast_scanningSkipped: Most apps are not deployed web services (libraries, CLIs, desktop apps)
✗dependency_update_automationNo Dependabot or Renovate configuration found
✓gitignore_comprehensive.gitignore properly excludes node_modules, dist, coverage, .DS_Store, *.log files
✗log_scrubbingNo log sanitization (no redaction support in logging or scrubbing documentation)
—pii_handlingSkipped: Not clear if apps process personal user data requiring PII handling
—privacy_complianceSkipped: Not clear if app processes personal user data requiring compliance
—secret_scanningSkipped: OSS evaluation mode - GitHub API returned 404 (not enabled or not accessible)
✗secrets_managementNo secrets manager integration (no AWS Secrets Manager, Vault, etc.), no .env.example template
Task Discovery0/3 (0%)
—backlog_healthSkipped: No open issues to evaluate backlog health
✗issue_labeling_systemNo open issues to evaluate labeling system (gh issue list returned empty)
✗issue_templatesNo issue templates in .github/ISSUE_TEMPLATE/ directory
✗pr_templatesNo PR template in .github/pull_request_template.md
Product & Analytics0/2 (0%)
✗error_to_insight_pipelineNo error-to-issue automation (no Sentry-GitHub integration or error tracking)
✗product_analytics_instrumentationNo product analytics (no Mixpanel, Amplitude, PostHog, Heap, or GA4)

start building

Ready to build the software of the future?

Start building

Arrow Right Icon