Our mission at Factory is to bring autonomy to software engineering in a way that prioritizes safety, responsibility, and transparency. While we continue to develop AI systems that can write, refactor, and deploy production-grade code, we recognize the potential for serious negative externalities—from deliberate misuse by malicious actors to unforeseeable accidents and mistakes.
This Safe Autonomy Readiness Policy (SARP) outlines our approach to:
- Identifying high-risk AI capabilities,
- Tracking potential vulnerabilities,
- Implementing safeguards to ensure our AI systems remain secure and transparent.
SARP applies to any AI system at Factory that can manage code, system operations, or privileged data. It is both a practical guide for day-to-day development and release planning, as well as a governance framework that influences our long-term vision for safety and responsibility.
Defining Frontier Capabilities
A system is considered under the purview of this policy when it can:
- Autonomously write or modify substantial amounts of code,
- Access or manage production codebases,
- Deploy changes across multiple environments.
Whenever a system significantly advances current capabilities in code generation, tool usage, or its degree of autonomy, we classify it as “frontier-level.” These systems demand additional risk evaluation and safeguards, particularly when they can independently push commits, modify system logs, or trigger builds without human intervention; access sensitive resources such as internal secrets, regulated user data, or critical infrastructure; and operate with expanded agency by taking long-term actions without immediate oversight or overriding their own boundaries.
Because these abilities introduce heightened security, reliability, and alignment risks, we apply a more rigorous level of auditing and mitigation to ensure frontier-level systems remain safe and beneficial.
Our Risk Management Framework
We take an iterative, layered approach to managing AI autonomy risks, focusing on four steps: Identify, Evaluate, Mitigate, and Monitor.
1. Identify
We begin by mapping vulnerabilities and attack vectors through threat modeling, capability audits, and user expectation analysis. By contrasting “intended” uses with what might be technically possible, we can spot gaps early.
2. Evaluate
Next, we stress-test our systems via red teaming, structured testing, and boundary testing. This phase ensures our guardrails are robust and that AI behaviors match the instructions we provide.
3. Mitigate
Where we uncover risks, we implement both technical (e.g., sandboxing, permission restrictions) and policy-based (e.g., requiring human approvals for certain actions) solutions. Our behavioral guidelines further clarify operational boundaries, creating an environment where autonomy is balanced with accountability.
4. Monitor
Finally, we evaluate and monitor the risk landscape—regularly monitoring new modela nd system capabilities to ensure our platform remain secure and reliable. This continuous feedback loop helps us refine our controls and integrate the latest threat intelligence.
Practical Examples of Immediate Concerns
While the concept of “AI alignment” can span broad, long-term questions, our daily practice focuses on tangible, near-term vulnerabilities.
Enterprise Controls & Oversight
In enterprise environments, granting developers free rein to adopt advanced, autonomous AI tools without robust top-down governance can quickly lead to chaos. Powerful autonomous systems that can write and deploy code pose unique risks, from inadvertent security breaches to intentional misuse. Without an overarching framework that enforces consistent standards, organizations risk exposing sensitive data, disrupting production environments, and losing track of potentially dangerous system changes.
At Factory, we believe these concerns demand an enterprise-wide approach to AI oversight. Your team can consolidate agent activity across teams and departments—under a single control plane, ensuring both real-time visibility and auditability. By enabling you to monitor every change made with Factory's AI systems, you gain clarity in how AI is transforming your organization.
Internal Governance
We have established a Safety Review to evaluate frontier-level AI systems and major AI updates, ensuring that any high-risk concerns are addressed before deployment. Key safeguards include:
- Incremental Rollouts: Each new feature enters limited internal testing before wider release, letting us observe stability and address emerging issues early.
- Incident Response & Transparency: In the event of a breach or incident, we immediately initiate a shutdown or rollback to protect systems and data, and promptly inform stakeholders of root causes and mitigations.
- Policy Evolution: Recognizing that AI capabilities evolve rapidly, we commit to revisiting this policy on a regular basis to incorporate learnings from the broader community and our own experiences.
Looking Forward
We aim to deliver powerful AI agents without risking catastrophic failures or data leaks. Our Safe Autonomy Readiness Policy ensures we proactively address day-to-day threats—like exfiltration and unintended system damage—while still acknowledging future alignment challenges.
We remain committed to updating these policies as our AI’s capabilities advance, and we welcome collaboration with users, partners, and the broader community to keep safety, reliability, and transparency at the heart of AI-driven software development.
For further inquiries or detailed security documentation, please reach out to Factory’s Security & Governance Contact - safety@factory.ai
