Module 5: Detection engineering and evaluation

Module 5: Detection engineering and evaluation#

Theme#

Detection engineering and evaluation

Essential Question#

How do we measure detection quality?

Module Components#

  • Book prose: conceptual framing, domain scenario, methods, and failure modes

  • Assignment: evidence-backed production of a specific artifact

  • Slides: presentation sequence for seminar or lecture delivery

  • Narration: spoken version of the slide flow

  • Rubric: criteria for evaluating the module artifact

  • Notebook: executable lab aligned with the module theme using synthetic security telemetry with login velocity, data transfer volume, process rarity, and threat labels

Module Artifact#

detection engineering packet with threat model, false-positive analysis, and triage workflow focused on detection engineering and evaluation: Create detection rules and evaluate false positives.

Professional Setting#

Students work as if advising a security operations center tuning AI-assisted detections before analyst rollout. Their work must be intelligible to SOC analyst, detection engineer, incident commander, and business system owner.

Use This Module in Order#

  1. Read the learning chapter.

  2. Review the slide deck with the matching narration.

  3. In Populi, open the private student-repository link for this course and enter modules/module-5.

  4. Clone the repository once or open its Codespace/Colab copy; run lab.ipynb and complete exercise.ipynb there.

  5. Self-check with the rubric, commit and push the work, then submit exactly what Populi requests.