High Precision Detections
Meaning, [nearly] all alerts are true positives. Hyperbole gratis: when the bells sound, the building’s already burned down. You achieve this via conservative thresholding: detections are tuned to require overwhelming evidence before firing. Cries of wolf are credible and urgent, but anything lurking amongst the weeds is missed.
Because the practical reality is that investigative bandwidth can’t be drawn down chasing ghosts and squeaky wheels. If you have salespeople on the floor, you’d want them moving cars, not entertaining entitled tire kickers.
If precision dips in pursuit of high-hanging fruit: productivity friction. Once you can’t easily discern sheep from wolves, the sheep will nervously overgraze, and the wolves are in a heavenly buffet of cover either way.
Low recall: as in, each net catches a subset of bad behavior, not all. But the denominator is the total potentially findable true positives--we can’t know for certain the exact prevalence across the enterprise. With precision, it’s possible to eventually triage which are false and derive the metric.
So, layer multiple precise detections that are each interpretable. A non-GMO ingredients list–sans “butylated hydroxytoluene.”
Non-human Insider Threat
Service accounts, APIs, and AI agents are the fastest-growing vector, outpacing human activity towards 50x. An autonomous workforce with persisted access, at machine speed and at scale means security only at the moment of exfiltration is dead.
Coined "agent behavioral analytics": additionally, even predominantly, monitor agent activity apart from human activity.
Insider risk is actor-centric (intent buckets to malicious, negligent, or impersonated) whilst emerging taxonomy for agents is behavior-centric (classifies observed deviations):
- Manipulated: input-level (prompt injection, poisoned context/tool output) that produces a one-off bad action without persistence.
- Subverted: hijacked, confused deputy
- Maliciously-configured: built to act against org interest
- Malfunctioning: bugs, flawed training
My repo demoing a detection ruleset over synthesized agent-tool logs and cards dataset.
- Action Hallucination: discrepancy btwn agent claim and tool call return, typically a failed/empty upstream. Flag if no tool calls in trace despite mention, or if result was error/timeout/empty.
-
Loops/Runaway Recursion: additive trio of token burn, repetitive sequences, 0% error rate (could be, respectively, legit big batch job, polling pattern, things are just peachy)
- Pattern hegemony = stuck
- Delegation cycles
- Deep session chains
- Cost: 3 std devs above baseline, see above for tightly correlated.
- Memory: cross-user/cross-tenant retrieval, keep secrets full stop, deletion anomalies (bursty/old-and-hot/first-ever).
-
Sequence/Rare Edge: loops catch repeating edges, sequence catch edges that shouldn't exist.
- Rare X-then-Y transitions given P(next tool | current tool)
- Taint-derived high risk: if current tool ingests untrusted content and the next is privileged.
-
Goal Drift: cosine similarity of normalized vectors of goal statement, planner step summaries, and final answer.
- Cliff: a single step has nothing to do with the goal.
- Slow slide: trajectory monotonically declines and cumulatively exceed baseline bound.
- Tool Health: flakiness can be symptom of hijacked endpoint.
- Latency: long-tail minority of tools can genuinely be slow-running and sit at high z, anchor to >99th percentile.
- Failure rate: at small sample sizes, raw percentages lie. 1 failure out of 2 calls is a coin flip. Wilson score gives confidence-interval-style est of 12% lower bound for 1-of-2 failures.
- Agent Card Drift during registration events.
- Impersonation: typosquat + shadowing/piggybacking
- Capability escalation: skill count jump
- Endpoint hijack + rug-pull: description diverged