Back to insights
Operations9 min read

Agent Incident Rollback for System-of-Record Repairs

Agent rollback is a compensating repair in the system of record. Replaying a checkpoint does not unpost overtime, unsend a notice, or restore a closed period.

agent rollbackincident compensationsystem of record repairAI agent operationsworkflow recovery

Direct Answer

Rollback repairs the record. Replay only restarts the agent.

Agent incident rollback is the compensating workflow that reverses or contains a side effect the agent already wrote to a system of record. The product of rollback is a repaired object: a reversed time-code, a withdrawn notice, a restored hold. It is not a replay of the model from an earlier checkpoint.

LangGraph can persist checkpoints, interrupt a run, replay from a prior step, or fork a new branch with updated state. Those primitives are useful for debugging and for choosing a better next action. The official time-travel note is explicit that updating state creates a branch and does not roll the original thread back. Even a clean replay re-executes later tool calls. It does not unwrite the payroll batch that already posted.

Our bias is to write a compensation catalog before the first write tool goes live. If the team cannot name the reverse action for a side effect, that action should stay behind a human or stay out of the agent.

False Comfort

A saved trace is not an undo button.

After the first production miss, teams often reach for the framework feature that looks like rewind. They find the checkpoint, change the prompt, and rerun. The new run may draft a better recommendation. The old run's write is still in payroll, the general ledger, or the employee notice queue.

Payroll timekeeping makes the gap concrete. An agent reclassified Saturday hours as overtime for a plant crew. The batch posted. Employees saw the preview. Finance closed the weekly estimate. Replaying the graph from the classification node does not pull the dollars back.

Replay re-executes work

LangGraph's time-travel docs state that nodes after the checkpoint run again, including API calls and interrupts. That can create a second write, not an undo.

Fork keeps the original history

A branched checkpoint is a new investigation path. The posted batch remains the production fact.

A pause is not a repair

Interrupts stop the next action. They do not compensate the last one.

The observability runbook at https://solzero.com/blog/agent-observability-runbooks-for-production-workflows tells operators what failed. Rollback tells them how to put the record back. Keep both. Do not rename one as the other.

Repair Catalog

A payroll timekeeping miss needs an explicit compensating action.

Stay with a plant payroll operations team. The systems of record are the time-and-attendance system, the payroll engine, the HRIS job and union rules, and the employee-notice channel. The agent may draft a reclass and route it. After approval, a tool may post a correction. Rollback starts when that post was wrong.

Identify the posted objects

Employee IDs, pay-period, earning codes, hours, and the batch or journal ID the tool returned.

Choose the legal reverse

A same-period correction, a next-period adjustment, or a void-and-reissue. The payroll calendar, not the agent, decides which one is allowed.

Check downstream consumers

Overtime already flowed into a labor report, a union export, or an employee preview. Each consumer needs a notify-or-repair step.

Preserve the original write

Do not delete the bad post from history. Compensation should be a new, attributable transaction so audit can see both sides.

Stop the lane

If the same reclass rule is still live, pause that tool before repairing a second crew.

Sequence

Contain, compensate, then file the fixture.

Handoff manifests at https://solzero.com/blog/agent-handoff-manifests-for-operational-workflows should carry the posted object IDs and the rollback note when the case leaves the agent. If the next owner has to rediscover the batch number, rollback has already started cold.

Approval packets at https://solzero.com/blog/approval-packets-for-human-in-the-loop-agents are the right surface for the compensating write. The first production repair should not be an unsupervised tool call.

Contain

Disable the write tool or freeze the earning-code lane. New runs can still draft. They cannot post.

Enumerate

List every object the bad run touched. A single employee preview is a different blast radius from a posted plant batch.

Compensate in the system of record

Use the payroll or time system's supported correction path. Do not invent a parallel ledger in the agent store.

Notify the people who already saw the error

Employees, supervisors, and finance partners who received the wrong preview need a correction notice, not a silent fix.

File a golden case

The miss should become a labeled fixture so the next prompt change cannot post the same reclass.

Framework Fit

Use durable state to inspect the miss, not to pretend the write never happened.

Checkpointers, interrupts, and time travel are still worth installing. They let an on-call engineer see the exact tool arguments, resume a paused approval, or fork a safer path after a human edit. They are incident forensics and control-flow tools.

The compensation catalog lives next to those primitives. For each write tool, name the reverse, the owner, the window in which reverse is possible, and the fallback when the pay period is closed.

Reversible now

Same-period time-code correction through the payroll API, with a reviewer.

Reversible later

Next-period adjustment after the week closes. The agent may draft it. Payroll still posts it.

Not reversible by tool

Tax filings, garnishment remittances, or bank files that already left the building. Those need a human process, not a retry.

A useful rule: if the reverse action is not already a first-class transaction in the system of record, the agent does not have a rollback. It has a hope.

Scoreboard

Measure unrepaired side effects, not traces replayed.

Rollback is working when a bad post is contained, compensated through the official path, and turned into a fixture before the next run. Useful measures include time from detection to lane freeze, objects repaired through the official correction path, unrepaired posts after the pay-period close, second writes created by naive replay, and repeats of a miss that already had a compensation note.

Do not treat a successful graph replay as recovery. If the dollars or the notice are still wrong, the incident is open.

The SolZero take is that agent rollback is payroll-grade change control applied to tool side effects. If a write tool is already live and nobody can name the reverse, start at https://solzero.com/#how-it-works.

FAQ

The question teams ask after the first bad post.

Can we just replay the checkpoint and overwrite the result?

Not for posted side effects. Replay may call the write tool again. Use the payroll correction path for the record that already exists, and use replay only to inspect or to draft the next action after the lane is frozen.

Further reading