Module 5: Detection engineering and evaluation

AINS6300 — AI in Threat Detection

Essential Question

How do we measure detection quality?

Scenario

a security operations center tuning AI-assisted detections before analyst rollout

Stakeholders: SOC analyst, detection engineer, incident commander, and business system owner

Core Moves

• Define the decision boundary

• Compare baseline and alternative

• Interpret evidence and assumptions

• Identify failure modes

• Recommend next action

Lab & Assignment

Create detection rules and evaluate false positives.

Artifact: detection engineering packet with threat model, false-positive analysis, and triage workflow focused on detection engineering and evaluation: Create detection rules and evaluate false positives.