AI SOC Triage and the Verification Tax

The vendor slide says your analysts are getting roughly a third of every hour back. One recent field trial put the number near 19 minutes per hour, and nobody in the room argued, because the agent really is closing alerts. It reads the phishing report, pulls the headers, checks the sender's history, detonates the attachment, and files a verdict before a human would have finished opening tabs.

Then you look at the calendar of an actual tier-two analyst three months into the deployment, and the recovered time is hard to find. She is not doing more threat hunting. She has not touched the detection backlog she was supposed to finally get to. Ask her where the 19 minutes went and she will tell you, a little sheepishly, that she spends a good chunk of them checking the agent's work.

Not because the agent is bad. Because she cannot see how it got there.

The verification tax

Here is the shape of the problem. The agent investigates an alert and produces a verdict: benign, expected behavior, closed. The verdict arrives in yet another pane of glass, a new console with its own login, sitting next to the SIEM, the EDR console, the ticketing system, and the team chat. The verdict comes with a summary. The summary is two paragraphs of fluent, confident prose.

And the analyst reads it and thinks: prove it.

So she re-runs the investigation. She queries the same logs the agent presumably queried. She checks the same user's sign-in history the agent presumably checked. Fifteen minutes later she agrees with the agent, closes the alert for the second time, and moves to the next one. The triage was automated. The trust was not. Call the gap between those two the verification tax, and notice that it is paid in exactly the currency the automation was supposed to return: analyst minutes.

The triage was automated. The trust was not.

Teams do not usually report this number, because it is embarrassing on both sides. The vendor does not want to hear that its verdicts get re-litigated. The SOC lead does not want to admit that the expensive automation is running in shadow mode with extra steps. But if you ask analysts directly, off the record, most will describe some version of the same routine. The agent closes the alert. The analyst quietly reopens the question.

Double-checking is the rational move

It is tempting to treat this as a change management problem. Give it a quarter, the thinking goes, and analysts will learn to trust the tooling the way they learned to trust sandbox verdicts and reputation feeds.

That misreads the incentives. An analyst who accepts a wrong benign verdict is not making an ordinary mistake. A phishing email dismissed as marketing spam, an impossible-travel alert waved through because the summary said VPN, a service account's 2 a.m. token grab explained away as automation: any one of those, if it turns out to be the first hop of an intrusion, becomes the finding on page four of the incident report. "The alert fired. It was closed as benign." Careers in this field are shaped by which alerts you closed, not which ones you escalated unnecessarily.

So when the reasoning behind a verdict is invisible, double-checking is not paranoia. It is the correct professional response to an unauditable claim. Analysts already apply this standard to each other. Nobody accepts "it's fine, I looked into it" from a colleague on a serious alert without asking what they looked at. The colleague answers by walking through the case: I pulled the OAuth grant, the app is registered to our own tenant, here is the change ticket from Tuesday. The verdict rides on the trail.

The agent, in most current deployments, offers the verdict without the trail. Or worse, it offers a trail-shaped summary: "sign-in activity was reviewed and found consistent with historical patterns." Reviewed how? Which window? Did it consider the MFA method change last week? A summary compresses the investigation into prose, and compression is exactly what an analyst cannot accept, because the alert that kills you lives in the details the summary smoothed over.

A summary is not an investigation

This is the distinction that decides whether auto-triage compounds or stalls: the difference between a summary of an investigation and the investigation itself.

An investigation, as any analyst who has worked a real case knows, is a sequence with structure. What did you check first, and why that first. What did the query actually return, not what you concluded from it. What hypotheses did you rule out, and on what evidence. Where did you stop, and what would have made you keep going. A senior analyst's case notes carry all of this. That is why a second analyst can pick up a half-finished case and continue it rather than restarting it.

A summary carries none of it. A summary is an argument that the work was done, written by the party that did the work. When the party is a human colleague with a track record, an argument is sometimes enough. When the party is an agent that was installed last quarter, is updated on the vendor's schedule, and fails in ways no one on the team has fully mapped yet, an argument is a request for faith. The analyst declines, reasonably, and pays the tax.

The fix is not a better summary. It is making the investigation itself the thing the agent hands over.

Inspectable in place

Picture triage where the agent's investigation is a thread the analyst can open, in the same workspace where the team already works, next to the case, not in a separate console.

The thread reads like a working analyst's notes, because that is what it is. Step one: pulled sign-in logs for the user, 30-day window, here is the query and here are the raw results, linked. Step two: flagged a new device enrollment nine days ago, checked it against the IT onboarding doc, matched, link to the doc. Step three: considered token theft, ruled it out because the refresh token predates the suspicious sign-in, evidence attached. Step four: verdict, benign, confidence noted, with the one remaining assumption stated plainly: this holds unless the onboarding record itself is compromised.

Now the analyst's job changes shape. She is not re-running the investigation; she is reviewing one, the way a senior reviews a junior's case work. She scans the sequence, spot-checks the step that matters (in this case, the device enrollment match), and either signs off in a minute or two, or replies directly on step three: "you ruled out token theft on timestamp alone, check whether the token was replayed from a second IP." The agent picks up the objection and continues the thread. The case record grows in place. Nothing is exported, summarized, or re-keyed into the ticketing system, because the thread is the case record.

That last part is the hinge. This only works when agents and analysts share a workspace, and the investigation state is the shared artifact between them. An agent that investigates elsewhere and reports back will always be delivering summaries, however detailed, because the analyst cannot touch the working state. An agent that investigates where the team lives leaves working notes the team can open, question, and extend. Verification collapses from re-doing into reading, and reading is cheap. That is when the 19 minutes start being real.

What the shared thread buys you later

Two second-order effects show up after a few months, and both are worth more than the minutes.

The first is training. Junior analysts have always learned the craft by watching seniors work cases: which log source to pull first, when a weird sign-in is worth an hour and when it is worth thirty seconds. As triage shifts to agents, that apprenticeship pipeline quietly breaks, unless the agent's investigations are readable. A good investigation thread is a worked example. A junior who reads forty of them in a month absorbs the same pattern library the seniors carry, including the cases where the agent got corrected mid-thread, which are the most instructive ones. Black-box triage starves your bench. Inspectable triage feeds it.

The second is audit. Today, reconstructing why an alert was closed eight months ago is archaeology: screenshots from a console that has since been upgraded, an export someone attached to a ticket, a analyst's memory. When the investigation thread is the case record, the answer to "why was this closed" is the thread itself, evidence links intact, objections and sign-offs included. The regulator's question and the analyst's question turn out to have the same answer, stored in the same place.

Do not let the dashboard pick the wrong number

A closing caution. Once auto-triage is in, the dashboard will offer you a seductive metric: time to dismissal. It will trend down and to the right, and it will be easy to report.

Resist making it the headline. A SOC optimized for dismissal speed is a SOC learning to close alerts faster, and the failure mode of triage has never been closing too slowly. It is closing wrongly. The number that tells you whether the automation is actually working is closer to this: what fraction of agent verdicts do analysts accept after reading the thread, without re-running the work, and does that fraction rise as the threads get better and the corrections get rarer.

When that number climbs, the recovered time stops leaking into re-verification and starts landing where you wanted it: the hunt backlog, the detection engineering, the tabletop exercises that keep getting bumped. The agent did not just give the hour back. The workspace made the hour spendable.