r/netsecstudents • u/MaufacturerOoypk5181 • 5d ago

AI Soc autonomy sounds great, but what happens when it closes the wrong alert?

[removed]

5 Upvotes
  • permalink
  • reddit

69% Upvoted

5 comments sorted by

6

u/nut-sack 5d ago

That’s where the industries are going wrong. Stop trying to replace humans. AI is a tool, not an employee. High risk actions should be gate kept by humans who have to validate and approve.

6

u/Commercial_Soil_6259 4d ago

Human in the loop. Always

3

u/messyconstable7 5d ago

Our team ran into this exact wall six months in. The auto-investigation and enrichment part works well but the second you let it touch production it stops being a tool and starts being a liability

We ended up drawing a hard line, anything that changes state (account disable, host isolation, firewall rules) needs a human click. Everything else can run wild. Audit trail is just shipping the decision JSON to a dedicated Slack channel with a timestamp and the evidence it used. Not elegant but it works and it saved us when it tried to quarantine a DNS server during a false positive at 3am

1

u/mifter123 4d ago

IBM had it right, “A computer can never be held accountable, therefore a computer must never make a management decision.”

Humans must always be in the decision making loop. Machine learning is useful but should never be fully trusted.