SAFE RFC: 11 Open Questions

SAFE specifies reporting timelines (72hr, 4-day, 30-day, 90-day), evidence requirements (prompts, traces, logs, configs, model versions, credentials, timelines), an 8-layer review framework, and remediation deliverables. Nowhere does it estimate what compliance costs.

Every requirement has a cost. The 72-hour notification requires real-time monitoring and 24/7 incident response. The evidence package requires comprehensive telemetry across the entire AI pipeline. The 8-layer review requires controls across model, instructions, safeguards, tools, environment, monitoring, human operations, and supply chain. The 30-day public report requires formal analysis from dedicated security engineers.

For large cloud providers, most of this infrastructure already exists. The marginal cost of SAFE compliance is small. For a small operator running open-weight models on their own hardware, every requirement is a new build — new logging, new monitoring, new incident response procedures, new security engineering capacity they may not have on staff.

Without a cost estimate, small operators can't budget for compliance. "Unknown" is the most expensive number in compliance. You either over-provision and waste money, or under-provision and fail the audit.

SAFE's evidence list assumes the reporting organization controls the full deployment stack. When an open-weight model is released and deployed by third parties downstream, the reporting responsibility is unclear.

The evidence SAFE requires includes prompts, traces, tool calls, logs, configurations, model and safeguard versions, permissions and credentials available during the run, agent and workload identities, a complete incident timeline, and reproduction testing. When a model developer releases open weights on HuggingFace, they don't control how those weights are deployed downstream. They don't control the prompts, the tool configurations, the credentials, the runtime environment, or the monitoring.

So who reports? The model developer can't provide the evidence — they don't have logs from a deployment they don't control. The downstream deployer may not have the infrastructure to collect it. If both report, the same incident produces two incomplete reports, neither of which satisfies the evidence requirements.

SAFE's control catalog will be produced by working groups populated by OSAA members. Several OSAA founding members sell the tools and platforms that SAFE's 8-layer review framework references by name: MDASH, Numbat, OpenShell, NOOA, Agent Guard, Lightwell.

This creates a structural conflict: the same organizations are defining the controls and selling the products that satisfy them. I've watched this pattern in other industries. The standard becomes a product requirement list. The vendors who wrote the standard have a head start because their products already "meet" it — because they wrote it to match what their products already do.

I'm not suggesting bad faith. I'm saying the incentive structure is what it is, and it should be acknowledged and addressed. Is there a recusal mechanism for vendors when their own products are being specified as reference controls? Will working group participation be public? Will there be a vendor-neutral control specification?

If the answer is that SAFE will address this in practice, I'd ask: where in the RFC is the mechanism? "We'll handle it" is not a structural safeguard — it's a promise. The whole point of a framework is to replace promises with structure.

SAFE requires reporting of "near misses, not only confirmed harm." The RFC does not define what constitutes a near miss or who determines whether an event qualifies.

Consider: an agent attempted a network call that would have reached an external system, but the sandbox blocked it. Near miss? An agent generated output that could have been used to exploit a vulnerability, but no human acted on it. Near miss? An agent was given credentials it shouldn't have had, but didn't use them. Near miss?

A large provider with a dedicated security team will have incident classification procedures. A small operator doesn't have that. They'll either report everything and drown in the volume, or report nothing and risk non-compliance. Without a defined threshold, the reporting burden is unpredictable — and unpredictable burden is the worst kind for small operators. You can't budget for "report everything that might maybe count." You can budget for "report events matching these specific criteria." One is a process, the other is anxiety.

Tier 2 shares "de-identified" advisories with exchange membership. AI incidents often involve unique configuration fingerprints — specific model versions, tool combinations, infrastructure patterns. Has SAFE defined a de-identification methodology?

"An agent running Model X v2.3 with Tool Y and Safeguard Z attempted unauthorized network egress." If only three organizations use that exact combination, the advisory identifies one of three. "A deployment using open-weight Model A fine-tuned on Dataset B experienced a tool permission failure." If the fine-tuned model is on HuggingFace, the organization that fine-tuned it is public information.

De-identification in healthcare has a methodology — remove the 18 HIPAA identifiers, apply expert determination. De-identification in AI incident reporting is harder because the technical details are the valuable part. You can't remove the model version, tool combination, and infrastructure pattern without making the advisory useless for learning.

In a coalition of 120+ members, the membership itself is the pool of suspects. The advisory doesn't need to name you — it just needs to narrow the field enough that the other members can figure it out.

SAFE states that regulators "retain their legal rights" and that learning is "separate from enforcement." The RFC does not describe what happens when a regulator compels disclosure of SAFE's confidential reports.

SAFE collects detailed forensic evidence: prompts, traces, logs, configurations, model versions, credentials, timelines. This is exactly the kind of evidence regulators want in enforcement actions. The RFC says learning is separate from enforcement, but that's a principle, not a legal structure.

If a regulator subpoenas the SAFE exchange, what happens? SAFE complies — then every confidential report is potentially discoverable. SAFE contests — on what legal basis? SAFE is a private framework, not a regulated entity. Its confidentiality promises are contractual, not statutory. SAFE has a legal opinion supporting non-disclosure — then it should be public, so members know what they're signing up for.

The gap between "we say it's confidential" and "we can legally protect it" is where trust lives or dies. A framework that can't answer this question is asking members to trust a promise, not a structure.

SAFE is framed as voluntary. The RFC does not address what happens when cyber insurers, procurement teams, or certification bodies require SAFE compliance as a condition of doing business.

I've watched this play out in other industries. A framework publishes as "voluntary." Large companies adopt it. Cyber insurers require it for coverage. Enterprise procurement teams write it into contracts. Certification bodies build programs around it. Within two years, it's not voluntary anymore — it's a condition of market access. The "voluntary" phase was just the runway.

The Cloud Security Alliance's own analysis of SAFE says: "a voluntary framework can still create de facto obligations if government contracting, procurement preference, or public trust" make compliance effectively mandatory. So SAFE's authors know this is how it works.

If SAFE knows this is how voluntary frameworks become mandatory — does it have a position on it? If not, is "voluntary" a description of the framework's intent or a term chosen for its marketing value? Silence is permission.

OSAA claims 120+ members. The GitHub repository tells a different story — one primary contributor, ten issues, eight unique authors. That's a 120:1 gap between claimed membership and actual participation. GitHub activity isn't the only measure of participation, but it's the only public measure available.

SAFE's requirements — 72-hour notification, full forensic evidence, 8-layer review, 30-day public reports — all assume enterprise-scale infrastructure. If the working groups that define the controls are made up of enterprise-scale organizations, the controls will be written for enterprise-scale organizations. Not because of bad faith, but because people write what they know.

A small operator, a self-hosted deployment, an independent researcher — these perspectives need to be in the room when the controls are written. Not after, when the controls are published for "public comment." During. In the working group. Voting on the language.

I've checked the public OSAA membership list. It reads like a who's who of enterprise tech. I didn't see a single organization I'd call small. If I'm wrong, correct me — I'd rather be wrong about this. But if I'm right, SAFE's controls are being written by large organizations for large organizations, and "voluntary" means "voluntary for the companies that can already afford to comply."

The 8-layer review framework names specific tools and platforms: MDASH, Numbat, OpenShell, NOOA, Agent Guard, Lightwell. These are named products from OSAA member companies.

If SAFE's control catalog says "organizations should implement safeguards equivalent to MDASH" — what does "equivalent" mean? Who determines equivalence? If I build my own safeguard system from scratch that achieves the same control objectives, is that compliant? Or do I need to use MDASH?

This is the difference between a standard and a product requirement list. A standard says "achieve this control objective." A product requirement list says "use this product." SAFE appears to be drifting toward the latter.

Without a vendor-neutral control specification, SAFE becomes a procurement list for OSAA member products. The distinction matters — it determines whether SAFE is a framework the whole ecosystem can use, or a framework that funnels business to a specific set of vendors.

The RFC does not describe what happens when a member leaves SAFE. Reporting obligations, data retention, and withdrawal of previously submitted evidence are all undefined.

Three questions any member should have answered before joining: If I'm a member and an incident occurs during my membership, then I leave SAFE, am I still obligated to complete the reporting cycle? The 30-day public report and 90-day remediation status may extend beyond my membership period — am I bound to them?

Can I withdraw reports or evidence I previously submitted to the exchange? If I join, submit incident reports, then realize the framework isn't working for me — can I take my data back? Or is it permanently in the exchange's database?

How long does SAFE retain member data? Is there a retention policy? If I leave, is my data deleted, archived, or retained indefinitely?

If members cannot withdraw their data, SAFE is a roach motel — data goes in but doesn't come out. Members are submitting detailed forensic evidence (prompts, traces, logs, credentials, timelines) to a shared database they can't withdraw from. That's a significant commitment, and the RFC doesn't acknowledge it. Members should know the exit before they enter.

SAFE's Evidence Preservation section specifies what evidence members must retain and provide, but does not require documentation of how that evidence was collected, transferred, stored, or handled between collection and submission. This is a chain of custody gap.

The gap is most acute for evidence that originates from third parties — cloud provider logs, evaluation partner telemetry, tool vendor audit trails, shared infrastructure — where the reporting member did not collect the evidence directly and cannot attest to its integrity from the point of origin.

The Supply Chain review layer asks: "Did a cloud, evaluation, data or tooling partner invalidate assumed controls?" This addresses whether a partner's controls failed. It does not address whether evidence received from that partner can be trusted to be authentic and unaltered.

When evidence comes from a third party, the reporting member is submitting evidence they did not collect. Without chain of custody, neither SAFE reviewers, regulators, nor affected parties can verify that the evidence originated from the stated source, has not been modified in transit, the third party's own logging is tamper-evident, or that each party who handled the evidence is documented.

This creates the same structural asymmetry as every other SAFE requirement. Large providers with dedicated forensics teams and vendor SLAs can implement chain of custody internally. Small operators who depend on third-party infrastructure they do not control cannot prove the integrity of evidence they did not collect themselves. The gap is survivable for enterprises and a compliance trap for everyone else.