The claim and the question it raises
A watchdog that only files issues still leaves the work to a person. Closing the loop means a system that detects a failure, reproduces it, writes a change, and proposes it — and every serious buyer hears that and asks the same thing.
Could a machine-written change reach my production system without a human in the path?
For a workflow that requires human release approval, check the effective permissions of the repair process and every deployment path it can reach. The absence of a merge function alone is insufficient if credentials, shell access, or CI triggers provide another route.
Control 01: isolate the repair checkout
Start from an isolated checkout of the deployed revision. Supply the recorded evidence needed to reproduce the finding instead of letting the repair process alter the running deployment.
A clone is a working-copy boundary, not a security boundary by itself. Check mounted volumes, credentials, shared services, and inherited permissions. Use sanitized inputs where production data is unnecessary.
The intended behavior is that a repair attempt leaves the deployed revision unchanged. Verify that behavior against the runtime and release configuration rather than assuming it from the directory layout.
Control 02: restrict execution and access
Run reproductions, tests, and replays in disposable environments with the minimum filesystem, credential, and network access they need. Disable network access when the test permits it; otherwise scope and record the required destinations.
Network restrictions remove some access paths but do not prove isolation on their own. Host mounts, container configuration, credentials, and the underlying runtime also matter. Test the enforced boundary rather than treating the word sandbox as a guarantee.
The differential replay is the substantive test: the candidate change is run against recorded historical runs to check that it fixes the reported failure without changing behavior on inputs that were previously handled correctly. A fix that repairs one class and silently breaks another is the normal failure mode of automated repair.
Control 03: produce a reviewable change proposal
Limit an automated repair attempt to a branch or draft pull request containing the reproduction, proposed diff, and evaluation results.
A draft status is a review signal, not a security control. Restrict merge and deployment permissions, inspect any CI triggered by the branch, and prevent the repair identity from changing its own protections.
A reviewer should be able to inspect, reject, or amend the proposal. Promotion to a running deployment must follow the approval policy configured for that workflow.
Review the effective permissions and reachable release paths of the repair identity, including repository automation.
Control 04: make approval responsibilities explicit
Separate the decision to attempt a repair, the review of its proposed change, and the authorization to release it. Assign accountable owners and record each decision.
Where the workflow requires distinct reviewers, enforce that separation with role and repository controls. Three steps do not automatically mean three independent people.
Check that the approved revision is the revision deployed. Include required checks, a rollback path, and a review of emergency bypass permissions.
Human review reduces risk when reviewers have relevant evidence, time, and authority. It remains fallible and depends on the surrounding access controls. No number of approvers provides an unconditional guarantee.
The same boundary applies to learning
A system that improves from feedback raises a quieter version of the same question: if it is learning, what exactly is changing, and can you get it back?
Prompts and curated reference data are useful places to capture reviewed corrections because their changes can be inspected and versioned. Test them against representative examples before promotion, using shadow evaluation when the workflow supports it.
Model or fine-tuning changes need their own version records, evaluation evidence, and recoverable checkpoints. Rolling back an artifact does not undo external actions already taken. Define recovery for the workflow's data and side effects as well as its code.
Technical examples explain design choices and their assumptions. Deployment controls and acceptance criteria are agreed for each workflow.