This is the uncomfortable shape of supply-chain attacks: you don’t have to be targeted. You just have to install something that depends on something that depends on something that got compromised. The attack doesn’t need to find you. It’s already inside the thing you trust.

For a homelab running Node.js services, this isn’t a corporate threat-model problem. It’s a you problem. Your machine has npm tokens, environment files, cloud credentials, maybe SSH keys sitting in ~/.ssh. You run things as your own user. If a postinstall script gets code execution on your box, it has what it needs. The attack surface is the whole machine, not just the package.

How These Attacks Actually Work

Before the case files, it’s worth having a clear taxonomy. These attacks aren’t all the same. The delivery mechanism matters, because it determines which defenses actually help.

The last row is what makes npm attacks particularly effective: installing a package isn’t a passive act. npm runs scripts. By default, any package in your tree can execute arbitrary code the moment you run npm install. You gave it permission when you hit Enter.

The Case Files

event-stream — 2018 — The Maintainer Handoff

The original, and still the most elegantly patient attack in npm history.

The event-stream package was a widely-used Node.js streaming utility with tens of millions of downloads. Its original author, Dominic Tarr, had largely moved on from maintaining it. In September 2018 he transferred ownership to a new contributor called right9ctrl, who seemed interested in helping out.

Two months later, right9ctrl released version 3.3.6. It added a new dependency: flatmap-stream. Buried inside flatmap-stream was an AES-256 encrypted payload that only executed if a very specific condition was met — the npm_package_description environment variable had to equal the string "A Secure Bitcoin Wallet". That’s the description field of a specific application: Copay, a Bitcoin wallet used by large holders.

The attack was inert for everyone except Copay users running versions 5.0.2 through 5.1.0. For them, it silently exfiltrated wallet credentials and private keys to a server in Kuala Lumpur. It sat undetected for about two months and amassed roughly 8 million downloads before a developer noticed the suspicious dependency.

The lesson: Targeted attacks can hide in plain sight. Your packages don’t have to target you specifically for you to be collateral damage. And “passing the maintenance baton” is a known attack surface — right9ctrl had literally asked for maintainer access.

ua-parser-js — October 2021 — Account Takeover

ua-parser-js detects browser, OS, and device type from User-Agent strings. Nearly 8 million weekly downloads. Used in production at Facebook, Microsoft, Amazon, Slack, Discord, and a few hundred other organizations you’ve heard of.

On October 22, 2021, someone compromised the maintainer’s npm account and published three malicious versions: 0.7.29, 0.8.0, and 1.0.0. Each shipped an install-time script (a preinstall hook) that, depending on the operating system, either downloaded an XMRig Monero cryptominer (Linux) or deployed a Windows cryptominer plus an infostealer that harvested credentials from the machine — then sent the haul home.

The malicious versions were live for roughly four hours. Clean versions (0.7.30, 0.8.1, 1.0.1) were published shortly after. Four hours was enough.

The lesson: Account takeover can happen to careful, legitimate maintainers. You have no visibility into the security posture of every account in your dependency tree. A lockfile pins the version but doesn’t protect you if the attack already exists in the pinned version — which is exactly why keeping lockfiles committed and auditing them matters.

node-ipc — March 2022 — Hostile Maintainer

This one is different, and in some ways more disturbing, because the attack came from the legitimate maintainer acting deliberately.

Brandon Nozaki Miller, who maintained the popular node-ipc inter-process communication package, pushed malicious code in node-ipc versions 10.1.1 and 10.1.2. The payload checked whether the machine’s IP geolocated to Russia or Belarus. If so, it recursively overwrote files with a heart emoji — effectively destroying data on those machines as a protest against the Russian invasion of Ukraine. (A separate, milder module the same author added around the same time, peacenotwar, merely dropped a protest text file; the destructive overwrite was node-ipc’s own code.)

Vue CLI, one of the most widely used JavaScript scaffolding tools, depended on node-ipc. The protestware shipped to users who ran Vue CLI setup scripts during that window.

The technical mechanism — a trusted maintainer pushing a destructive payload — is identical to a supply-chain attack. That the motivation was political rather than financial changes nothing from a systems security perspective. Code you didn’t write and can’t audit executed on your machine and destroyed data.

CVE-2022-23812 was assigned. The malicious versions were pulled.

The lesson: Behavioral auditing of packages matters, not just version pinning. A package that was safe last week can become dangerous this week because of a human decision, not a hack. This is the argument for tools that flag behavioral changes in packages — new network calls, new file operations, new lifecycle scripts — between versions.

September 2025 — The qix Compromise (chalk, debug, and 16 others)

On September 8, 2025, malicious versions of 18 npm packages started appearing on the registry. The packages included chalk (around 300 million weekly downloads), debug (even higher), color-convert, color-name, ansi-styles, strip-ansi, and a dozen others — collectively over two billion weekly downloads.

The attack vector was a phishing campaign targeting npm maintainers. The domain npmjs.help was used to harvest credentials through a convincing fake “two-factor authentication update” email. The account compromised belonged to qix, a prolific maintainer who controlled a substantial slice of the color and terminal utility ecosystem — packages that show up in virtually every Node.js project’s transitive dependencies.

The malicious payload hooked into fetch, XMLHttpRequest, and browser wallet APIs (window.ethereum, Solana endpoints) to intercept and reroute cryptocurrency transactions to attacker-controlled addresses. A classic browser crypto-clipper — but delivered via packages that run in Node.js build pipelines, which also get bundled into browser-facing code.

Aikido Security detected the malicious versions approximately two hours after they appeared. During that two-hour window, the malicious code reached an estimated one in ten cloud environments.

The lesson: Ubiquitous utility packages — the ones every project pulls in — are the highest-value targets precisely because they’re everywhere. A two-hour exposure window on a package with hundreds of millions of weekly downloads is a very bad two hours.

Shai-Hulud — September–November 2025 — The Worm

This is the one that represents a genuine evolution, and if you run a homelab with any npm activity, it’s worth understanding in detail.

Shai-Hulud (named after the sandworm from Dune, because threat researchers have taste) was first detected by ReversingLabs on September 15, 2025, with the initial infection beginning September 14. What made it different from every previous npm attack is what it did after it ran on your machine.

Most supply-chain attacks are one-directional: compromise a package, get code execution on victim machines, exfiltrate credentials. Done.

Shai-Hulud added a second stage: it used the credentials it stole to spread.

The worm’s lifecycle:

1. Victim runs npm install on a poisoned package
2. Postinstall script runs, downloads and executes TruffleHog — the open-source secret scanner — against the victim’s local filesystem and environment
3. TruffleHog finds npm tokens, GitHub tokens, AWS credentials, GCP credentials — whatever is present
4. Credentials are exfiltrated to attacker-controlled webhooks
5. If a valid npm publish token is found: the worm publishes malicious versions of every other package the victim maintains
6. Each newly-infected package repeats from step 1 when anyone installs it

Over 180 packages were compromised in the initial wave. A second variant (“Shai-Hulud 2.0”) arrived in November 2025 with improved evasion techniques and compromised over 500 packages. A further campaign dubbed “Mini Shai-Hulud” surfaced in spring 2026, hitting packages across the TanStack, Mistral AI, and other dependency trees.

The self-replication is what makes this qualitatively different. Previous attacks required the attacker to maintain access to a specific compromised account. Shai-Hulud spread the infection autonomously using its victims as propagation infrastructure.

The lesson for a homelabber: If you maintain any npm packages — even tiny personal ones — your npm publish token sitting in ~/.npmrc is a worm propagation vector. Your machine is a potential node in someone else’s botnet. Least-privilege tokens with minimal scope and short TTLs aren’t paranoia; they’re hygiene.

The Defense Stack

These aren’t in alphabetical order. They’re in rough priority order for a homelab that runs Node.js services. Do the cheap ones first.

1. Commit your lockfile. Always use npm ci.

If you take nothing else from this post, take this.

npm install resolves versions at install time and can silently pull in newer versions of packages. npm ci installs exactly what is in your lockfile and fails if there’s a mismatch. It is strictly safer for anything that isn’t an initial dependency resolution.

# In any CI/CD pipeline or automated deploy:
npm ci

# Not this:
npm install

Your package-lock.json (or yarn.lock, or pnpm-lock.yaml) must be committed to your repository. A lockfile that isn’t committed isn’t protecting anything. It only pins versions if it’s there.

This doesn’t prevent an attack that’s already in the version you’ve locked to — but it prevents you from silently pulling in a compromised newer version because someone ran npm install and auto-updated.

2. Disable lifecycle scripts for CI installs

npm runs preinstall, install, and postinstall scripts by default. These are how most supply-chain attacks get code execution. You can disable them:

# One-time config:
npm config set ignore-scripts true

# Or per-command:
npm ci --ignore-scripts

The cost: some legitimate packages use postinstall scripts for binary compilation (native addons, things like esbuild, sharp, node-sass). With scripts disabled, those packages won’t work until you run their scripts manually. This is annoying; it is also a meaningful security improvement. You can selectively re-enable for specific packages:

# After confirming the package is legit:
npm rebuild esbuild

For a homelab CI pipeline that just runs your services, the odds are good that --ignore-scripts works without any extra steps. Try it.

3. Run npm audit — but understand its limits

npm audit
npm audit --audit-level=high

npm audit checks your dependency tree against the npm advisory database for known vulnerabilities. It’s free, it’s built in, and you should be running it. The limit is that it only catches what’s been reported. Shai-Hulud packages were live for days or weeks before they were in any advisory database.

For broader coverage, Google’s OSV-Scanner checks against multiple vulnerability databases simultaneously:

# Install (it's a Go binary — Homebrew, Go, or a release binary):
brew install osv-scanner
# or: go install github.com/google/osv-scanner/v2/cmd/osv-scanner@latest

# Scan a project directory (or point at a lockfile):
osv-scanner scan source .
osv-scanner --lockfile package-lock.json

Run both in your pipeline.

4. Use Socket.dev for behavioral analysis

Socket.dev is the most useful tool in this space that most people haven’t heard of. Where npm audit checks against known CVEs, Socket analyzes package behavior — does this new version make network calls it didn’t make before? Does it access the filesystem in new ways? Does it have a new postinstall script?

This is exactly the kind of signal that would have flagged event-stream and ua-parser-js earlier. The free tier covers open-source projects; there’s a GitHub integration and a CLI:

npx socket scan .

For a homelab context: I run this before pulling in any new dependency, not as part of automated CI. The noise-to-signal ratio is better when it’s a human check on a specific decision rather than a gate that blocks every install.

5. Pin critical dependencies and review diffs before updating

Dependabot and Renovate will auto-create PRs when your dependencies have updates. This is good. Auto-merging those PRs without review is not.

# .github/dependabot.yml — reasonable config
version: 2
updates:
  - package-ecosystem: "npm"
    directory: "/"
    schedule:
      interval: "weekly"
    # Don't auto-merge major version bumps
    open-pull-requests-limit: 10

When a PR comes in from Dependabot: look at the diff. Specifically look at whether the update changes any lifecycle scripts, adds new dependencies, or modifies network behavior. For patch updates on packages you’ve used for years this can be a 30-second check. For major version bumps it deserves more time.

The alternative is a fully automated pipeline where a compromised version update merges itself and deploys to your services. Don’t build that pipeline.

6. Generate and check npm provenance

As of npm 9.x, packages can be published with sigstore-backed provenance attestations that link a published version to a specific Git commit in a specific repository, via a verifiable build chain. When provenance is available:

# Verify registry signatures and provenance attestations for your installed tree:
npm audit signatures

# Or check a single package on the npm website — look for the "provenance" badge

Not every package has provenance yet — it requires the publisher to opt in. But for critical packages in your stack, it’s worth checking. A package with attestable provenance is meaningfully harder to backdoor than one without. This is early-stage infrastructure but it’s moving fast.

7. Use short-lived, scoped npm tokens

If you publish packages or if your CI/CD pipeline needs to read private packages:

# Create a scoped token (requires npm account; granular tokens are also available in the npm web UI):
npm token create --read-only        # for installs only
npm token create --cidr=10.0.0.0/8  # restrict token use to an IP range

Tokens with publish scope should have short TTLs and be rotated regularly. If Shai-Hulud runs on your machine and finds an npm token, the scope of that token determines how much damage it can do. A read-only token can’t publish. A publish token with a 24-hour TTL buys you a detection window.

Store tokens in your secrets manager, not in .npmrc files committed to source control or left in plaintext in environment variables. (If you’re using a homelab secrets manager — Vaultwarden, HashiCorp Vault, whatever — npm tokens belong there.)

8. SBOM generation

Software Bill of Materials — a machine-readable inventory of every component in your build — is increasingly a compliance requirement for serious projects and useful for homelabs that care about what they’re actually running:

# Generate an SBOM in CycloneDX format:
npx @cyclonedx/cyclonedx-npm --output-file sbom.json

# Or in SPDX format:
npx spdx-sbom-generator

This is more useful for auditing what you have than for real-time attack prevention — but it gives you a baseline. When an incident occurs, you can answer “did we depend on that package?” in seconds rather than hours. I’ll cover SBOMs in more depth in a future post.

Cross-reference

If you’re thinking about detection for supply-chain attacks — catching malicious network calls after the fact, spotting TruffleHog running where it shouldn’t — that’s covered in the detection engineering post: /04-detection-engineering/. And if you want to understand how threat intelligence fits into knowing which packages to watch before they show up in advisories, that’s /03-threat-intel/.

The Takeaway

The honest version of this:

Commit lockfiles and use npm ci. This is free and takes ten minutes to set up. It doesn’t prevent attacks in your pinned version but it stops you from silently drifting into a compromised one.

Run scripts off by default in CI. The attack surface for supply-chain attacks in npm is the postinstall hook. Take it away where you can.

Review dependency updates before they merge. Not every update, not every line — but someone with eyes should look at what changed before a new version hits your running services. Automated merging of a compromised update is how a two-hour window of malicious package availability becomes a persistent compromise.

Treat your npm tokens like passwords. Scoped, short-lived, in a secrets manager. If Shai-Hulud finds a read-only install token, it can steal it. If it’s also a publish token, it becomes a propagation vector.

You can’t audit every dependency in your tree. The maintainer graph is too large and too distributed for that to be realistic. What you can do is make compromise harder to silently land (lockfiles, scripts off), make it more likely you notice when something changes (Socket.dev, diff reviews), and limit what an attacker gets if they do reach your machine (scoped tokens, secrets hygiene).

Supply-chain attacks are loud at the ecosystem level and silent at the individual machine level. The defenses above are about flipping that ratio: making them louder where you can actually see them.

This post is part of the Dispatches track — topical deep-dives outside the main “Start Your AI Homelab” series.

That’s automation work, not analyst work. SOAR is how you take it back.

This post covers Security Orchestration, Automation, and Response at the home-lab scale — what it actually is, what to automate first, and the guardrail you must not skip. Because you can absolutely automate your way into deleting a running service or locking yourself out of your own accounts. That’s a fun story to tell later, less fun to live through.

What SOAR Actually Means

SOAR is three things bundled into one acronym, and they’re worth separating because they have very different risk profiles:

Security Orchestration — connecting your tools so they can talk to each other. Your SIEM sends an alert to your case management system, which calls your threat intel platform, which writes results back to the case. Orchestration is plumbing. It’s low-risk and high-value.

Automation — replacing manual steps with scripts. Instead of you tabbing over to VirusTotal and copy-pasting a hash, a workflow does it in two seconds and writes the result into the alert record. Automation is also generally low-risk when it stays read-only.

Response — actually doing something to your environment in reaction to an alert. Blocking an IP at the firewall. Disabling a user account. Isolating a host from the network. This is where risk gets real.

Most SOAR tutorials gloss over this distinction and treat it like a smooth continuum. It’s not. The gap between “look up a reputation score” and “kill a running session” is significant, and how you handle that gap determines whether automation helps you or causes its own incident.

At home, you’re running SOAR with a team of one. That means there’s nobody else to check your playbook before it runs, and “I’ll just automate the response too” is a shorter path to a self-inflicted outage than you’d think.

What to Automate First

The highest-leverage automation targets are the things that are high-volume, low-stakes, and identical every time. In practice that’s four categories:

Enrichment — every alert that mentions an IP, domain, file hash, or email address needs the same context: is it known-bad, where is it geographically, when was it last seen in threat intel, is the asset it touched important? Automating enrichment means every alert arrives with that context already filled in. Your triage time drops from ten minutes to thirty seconds because the legwork is done.

Alert triage and deduplication — if the same external scanner hits your SSH port forty times in an hour, that’s one event, not forty alerts. Automation can correlate and deduplicate before anything reaches your queue. A clean queue is a queue you’ll actually work.

Notifications — alerting the right person (in a home lab, that’s you) in the right channel at the right time. A high-severity alert at 2 AM probably wants a push notification. A low-severity log anomaly probably wants a daily digest. Routing decisions like these are easy to codify.

Case and ticket creation — when something does need human attention, the case should already exist with enrichment attached by the time you look at it. Creating cases manually is overhead that adds no analytical value.

Clear-cut containment — the narrowest category, and the one with the most caveats. A small number of automated responses are safe enough to run without human review: adding an IP to a watchlist, dropping a connection at the network perimeter when a feed has already confirmed it’s active C2 infrastructure. But the line here is narrow. More on that in the next section.

The other 20% — the cases that need judgment, the alerts with ambiguous context, the response actions that affect running services — those need a human. Automation should be buying back time so that human can think, not trying to replace the thinking.

The Playbook Concept

A playbook is just a documented, executable workflow: given trigger X, do steps A, B, C, then record what happened.

The basic structure is consistent regardless of the tool:

1. Trigger — something fires. An alert from your SIEM, a new entry in a threat intel feed, a threshold crossed in your metrics, a file landing in a watched directory.
2. Enrich — gather context automatically. IP reputation, domain age, hash verdict, asset criticality, related historical events.
3. Decide — based on what enrichment found, route the alert. Is this obviously benign? Close it with a note. Is this high-confidence malicious? Escalate. Everything else: queue for review.
4. Act — execute the appropriate response. Read-only (log, notify, create a case) or active (contain, block) — more on this shortly.
5. Record and feed back — write outcomes back to the case, close the loop. What happened, what was decided, what was done, by whom or by what. This record is what you use to tune the playbook later.

The record-and-feedback step is where most home implementations cut corners, and it’s the one that compounds. Playbooks that don’t record outcomes can’t improve. Six months in you have no idea whether the enrichment step is actually informing decisions or just adding noise.

Tools Worth Knowing

This is the part where I give you a table instead of a wall of text, because your home threat model doesn’t need a 3000-word comparison of enterprise SOAR platforms.

The MISP integration from part 3 fits here naturally — Cortex has a MISP analyzer, and Shuffle has native MISP actions. Once your threat intel platform is connected to your case management and your case management is connected to your automation layer, enrichment starts flowing without manual steps.

For a home lab starting point: n8n for orchestration and notification plumbing, TheHive for case management, and Shuffle when you want to build more structured playbooks. You don’t need all three on day one.

A Worked Playbook: Anomalous Login Alert

Here’s a concrete example end to end. The scenario: your detection rule from part 4 fires on a login from an unusual source IP — either a new location, an unusual hour, or an IP that doesn’t fit the normal pattern for this account.

Step 1 — Alert fires, playbook triggers. The SIEM (or log analysis tool, or whatever you’re running) generates an alert with: source IP, account, timestamp, source application.

Step 2 — Auto-enrich the source IP. The playbook immediately fires off parallel lookups:

• ThreatFox / OTX for known-bad reputation
• Geo IP lookup for country, city, ASN
• Your internal asset inventory: is this IP in your known-good list?

This takes three to eight seconds. No human involvement.

Step 3 — Evaluate travel plausibility. If you have a previous login on record, the playbook checks: could this person physically be in both locations given the time delta? A login from your home city and then Toronto two hours later — plausible. A login from home and then Singapore four minutes later — not plausible.

Step 4 — Risk decision.

If the IP is in a threat feed or the travel is impossible: the alert is classified high-risk. Proceed to Step 5.

If the IP is clean and the travel is plausible: lower priority. Log the event, attach the enrichment, close the alert with a note. No human required.

Step 5 — Open a case, notify. TheHive gets a new case created automatically, with all enrichment attached. You get a push notification with a summary: source IP, reputation verdict, geo, the travel-plausibility result, and a link to the case.

At this point, you are informed and a case exists. Nothing destructive has happened yet.

Step 6 — Human decision gate. The case in TheHive shows two action buttons: Approve containment or Dismiss as false positive.

If you dismiss it: the case closes, the enrichment is logged, the playbook records your decision.

If you approve: the playbook runs the response — in this example, ending the active session for that account and flagging it for password review. Because you clicked a button to authorize it, you knew it was coming and can reverse it if needed.

Step 7 — Record outcome. Whatever happened — dismissed, confirmed, contained — gets written back to the case. Time from alert to decision, who acted, what ran. This is your audit trail and your playbook improvement data.

The gate at Step 6 is the entire point. Enrichment runs automatically. Notification runs automatically. Containment does not run automatically. A human was in the loop for the step that mattered.

The Flowchart

flowchart TD
  A[Alert: anomalous login] --> B[Auto-enrich: IP reputation, geo, asset]
  B --> C{Known-bad or high-risk?}
  C -->|No| D[Lower priority + log + close]
  C -->|Yes| E[Open case in TheHive + notify]
  E --> F{Is the response reversible?}
  F -->|Read-only / soft contain| G[Auto-respond]
  F -->|Destructive / irreversible| H[Hold for human approval]
  H --> I[Analyst approves]
  I --> J[Execute response]
  G --> K[Record outcome + feedback]
  J --> K

Closing the Loop — The Whole Series

Here’s what the five disciplines look like when they’re connected:

Automation buys back the time you were spending on repetitive triage. You spend that time hunting — going looking for things your rules haven’t caught yet, the way we covered in part 2. The things you find while hunting become detections — you take the pattern, convert it to a rule, and codify it, which is what part 4 was about. Those detections fire alerts. Automation triages the alerts. And so it runs.

The five disciplines aren’t a checklist you work through once. They’re a loop that feeds itself. Better detections generate cleaner alerts. Cleaner alerts make automation more reliable. More reliable automation frees analyst time. Analyst time goes to hunting. Hunting improves detection.

That loop is what a mature security practice actually looks like, regardless of whether it’s running in a home lab or an enterprise. The tools are different. The logic isn’t.

The map at the start of this series laid all five disciplines out before we built any of them. Go back and read it now if you want — it hits differently once you’ve worked through each post.

Where to Go From Here

This is the end of the Blue Team Homelab series, but it’s not the end of the build.

If I were starting fresh with what I know now, I’d pick one of these five disciplines — whichever connects most directly to a real problem you have today — and go deep on it before touching the others. Not because the others don’t matter. Because shallow coverage of all five teaches you less than genuine fluency in one, and fluency in one pulls you naturally into the next.

The right starting point for most people is detection engineering. Writing a rule and watching it fire on something real is the fastest way to understand why all the other pieces exist. From there, threat intel and hunting expand your scope. Automation scales what you’ve built. But you have to have something worth scaling first.

Pick the thread that pulls hardest. Follow it.

The series starts at Why a Home SOC? if you want to send someone to the beginning.

Previous: Detection engineering: detections as code

Detection engineering is the discipline of treating detections like software. Not as one-off queries you configure through a UI and hope for the best, but as a durable product with a lifecycle: authored, reviewed, tested, deployed, tuned, and eventually retired. Everything that makes software maintainable — version control, peer review, automated testing, documented intent — applies here too.

This post works through what that means in practice for a home blue team setup.

What detection engineering actually is

The term sounds more enterprise than it is. The core idea is simple: a detection is code, and code should be treated like code.

That means:

• It lives in version control. You can see every change, who made it, and why. You can roll back if a change causes noise.
• It has documented intent. Not just what it looks for, but why — what adversary behaviour it maps to, what it assumes about the environment, what it’s known to miss.
• It’s tested before it goes live. Not “I ran it and it didn’t error.” Tested as in: someone intentionally triggered the condition and confirmed the detection fired.
• It has an owner. Detections don’t maintain themselves. Someone is responsible for knowing whether it still makes sense six months from now.

The alternative — clicking a detection into a SIEM console with no documentation, no test, and no review — produces a pile of rules that nobody understands well enough to tune. When the environment changes, nobody knows which detections will break. When something fires at 2 AM, nobody remembers what the rule was supposed to catch.

For a home lab, the stakes are lower, but the discipline is worth building. Learning to do it right when the consequences are minor means you’ll do it right when the consequences aren’t.

Detections as code

The practical implementation of detection-as-code looks like this:

Every detection lives in a git repository. A directory per detection, a YAML file with the rule, and a short document explaining what it is and why it exists. PRs to add or change detections get reviewed before they’re merged. The merge history is an audit trail.

CI runs on every PR. At minimum, a syntax check. If you’re using Sigma (more on that below), CI converts the rule to your SIEM’s query language and confirms it compiles without error. If you’ve written unit tests — input events that should and shouldn’t match — CI runs those too.

Deployment is automated from main. Merging to main triggers a sync to your SIEM. You never manually paste a rule into a console. The console reflects exactly what’s in git, no more and no less.

Everything is auditable. A detection that exists in the SIEM but not in git is a defect. A rule edited directly in the console is a defect. The git repo is the source of truth, and anything that diverges from it is wrong.

This sounds heavyweight for a homelab. In practice, you can implement a lightweight version with a single git repo, a GitHub Actions workflow that runs sigma-cli on push, and a script that syncs rules to your SIEM on merge. The friction of doing it this way is low. The payoff — knowing exactly what you’re running and why — is real.

Sigma: write once, convert anywhere

Sigma is a generic, vendor-agnostic detection rule format. You write a rule in YAML once, and a converter tool translates it into the query language your specific SIEM understands — whether that’s Splunk SPL, Elasticsearch Query DSL, OpenSearch, QRadar, or something else. This matters because it means your detection library isn’t married to one product.

A Sigma rule has a handful of required sections: metadata about the rule (title, status, description), a logsource block that says what kind of events this rule applies to, a detection block that defines what to look for, and metadata like level (low, medium, high, critical) and tags to map to MITRE ATT&CK techniques.

Here’s a small, real example — detecting the kind of local reconnaissance that shows up early in many intrusions: a user running whoami, net user, or net localgroup in quick succession.

title: Suspicious Local Recon Commands
id: 7b3d1a0f-912c-4e8d-a9b2-c5f87d6e2301
status: experimental
description: >
  Detects sequential execution of common local reconnaissance commands
  used to enumerate the current user, local accounts, and group memberships.
  Typically seen in the early stages of post-exploitation.
references:
  - https://attack.mitre.org/techniques/T1033/
  - https://attack.mitre.org/techniques/T1069/001/
author: MrK
date: 2026-08-14
tags:
  - attack.discovery
  - attack.t1033
  - attack.t1069.001
logsource:
  category: process_creation
  product: windows
detection:
  selection:
    CommandLine|contains:
      - whoami
      - 'net user'
      - 'net localgroup'
  condition: selection
falsepositives:
  - Legitimate sysadmin activity
  - Automated IT management scripts
level: low

A few things worth noting about this structure. The logsource section — category: process_creation, product: windows — tells the converter which data source this applies to. The detection.selection defines what a matching event looks like; condition: selection means “alert when any event matches the selection.” For more complex rules you can combine multiple named selections with boolean logic: condition: selection1 and not filter.

The falsepositives section isn’t enforced by the tooling. It’s documentation for whoever is looking at an alert at 2 AM. That matters.

To convert this to an Elasticsearch query:

# Install sigma-cli and the Elasticsearch backend
pip install sigma-cli
sigma plugin install elasticsearch

# Convert the rule
sigma convert -t elasticsearch -p ecs-windows recon-commands.yml

The output is a query you can paste directly into Elastic. If you’re using Splunk, swap -t elasticsearch for -t splunk. The rule itself doesn’t change.

The detection lifecycle

A detection has a life beyond the moment you write it. Understanding the full arc changes how you build them.

Idea / hypothesis. Most good detections start with a question: “if an attacker did X, what would that look like in our logs?” The source might be a threat hunt (you found something suspicious, and now you want an automated signal for it — see the threat hunting post), or a piece of threat intelligence (a TTP is associated with a relevant threat actor — see the intel post), or a post-mortem from an actual incident. This is the hypothesis stage: you have a theory about what bad looks like.

Develop. Turn the hypothesis into a detection rule. Write the Sigma YAML. Write the intent documentation. Write down what you think it will miss and what you expect to generate false positives on.

Test. Before this goes anywhere near production, you need to confirm it actually works. More on this below. Skip this step and you have a hypothesis masquerading as a control.

Deploy. Merge to main. The CI/CD pipeline converts and syncs it to your SIEM. The detection is now live.

Tune. The first weeks in production reveal the false positive rate. Legitimate admin scripts that match the pattern. Monitoring agents that look suspicious. Each false positive is a tuning opportunity: add a filter, tighten the condition, document the known-good pattern. Every tuning change goes through the same PR process.

Retire. The technology being monitored was decommissioned. A better, broader detection superseded this one. The false positive rate is unmanageable and the signal-to-noise ratio never improved. Not every detection earns indefinite production status. Retiring a detection is a deliberate decision, not benign neglect — and it’s healthier than keeping a noisy rule that everyone learns to ignore.

The ADS framework

Palantir published the Alerting & Detection Strategy (ADS) framework as a structured template for documenting what a detection is and why it exists. The idea is that every detection should have a corresponding ADS document, and that document should contain enough context for a stranger to understand, maintain, and eventually retire the rule.

The sections:

Goal. One or two sentences. What adversary behaviour or event does this detection target?

Categorization. The MITRE ATT&CK technique (or techniques) this maps to. Not because ATT&CK is gospel, but because it’s a shared vocabulary that tells you where in the kill chain this fires.

Strategy Abstract. How does the detection work? What data source does it use? What does a matching event look like?

Technical Context. What do you need to know about the environment for this detection to make sense? What logging must be enabled? What systems must be generating the relevant events?

Blind Spots and Assumptions. What does this detection miss? If the attacker runs the same recon commands as a service account that’s in your allowlist, this won’t fire. Say so. If this assumes Windows event ID 4688 is enabled, say so. Making the blind spots explicit is more valuable than pretending they don’t exist.

False Positives. Known benign conditions that will match. Sysadmin activity, monitoring agents, automated scripts. The more specific the better — “IT management scripts” is less useful than “the Ansible playbook baseline-audit.yml generates this event on every run.”

Validation. How was this detection tested? What tool, what technique, what result? This section closes the loop on whether the detection actually fires.

Priority. The severity level when this fires and how it should be treated in triage.

Response. What should happen when this fires? Not an exhaustive runbook, but the key questions to answer first.

Writing an ADS document for every detection takes maybe fifteen minutes. What it buys you is a detection that someone else — or future you — can actually maintain. A rule file with no documentation is a black box. With the ADS template it’s a defensible artifact.

Testing your detections

This is where most home blue teamers skip a step, and it’s the most important step in the whole pipeline.

Atomic Red Team is a library of small, targeted tests — “atomics” — mapped directly to MITRE ATT&CK techniques. Each atomic is a minimal, documented way to trigger a specific behaviour. Run the atomic, check whether your detection fired. If it didn’t fire, you have a hypothesis, not a control.

# Install Atomic Red Team (PowerShell, on a test Windows machine)
IEX (IWR 'https://raw.githubusercontent.com/redcanaryco/invoke-atomicredteam/master/install-atomicredteam.ps1' -UseBasicParsing)

# Run the atomic for T1033 - System Owner/User Discovery
Invoke-AtomicTest T1033 -TestNumbers 1

# Run the atomic for T1069.001 - Local Groups
Invoke-AtomicTest T1069.001 -TestNumbers 1

If your detection fires: promote the rule to production. Document the validation result in the ADS document. If it doesn’t fire: go back to the rule. The log source might not be generating the expected event. The query logic might be wrong. Something is missing. The atomic doesn’t lie — if the behaviour happened and the detection didn’t fire, the gap is in your detection, not in the test.

CALDERA is a step up: a full adversary emulation platform where you define an operation (a sequence of ATT&CK techniques) and let it run against a test host, then check how many steps your detection coverage caught. This is overkill for a first homelab detection setup, but it becomes useful once you have enough rules to want to measure coverage systematically.

The important mindset shift: testing isn’t optional polish at the end of the process. It’s the step that determines whether you have a detection or a guess. Run the test before you put the rule in production. Every time.

False positives are the real enemy

A detection that fires accurately on malicious behaviour is useful. A detection that fires on everything is noise, and noise is how alerts become something people learn to click through without reading.

False positive tuning is ongoing, not a one-time activity. When a detection fires in production and the activity turns out to be benign, that’s not a failure — it’s data. Add the filter. Document the known-good pattern in the ADS. Retest. Merge.

The thing to watch for is detection hoarding: keeping noisy rules active because they feel safer than retiring them. A detection with a 99% false positive rate costs your future self more attention than it’s worth. If a detection consistently produces noise and you’ve tried to tune it and it’s still noisy, retire it. Write a better one if the underlying hypothesis is still valid.

The metric that matters isn’t “how many detections do we have.” It’s “how often does a detection fire on something worth looking at.” Fewer high-fidelity rules beat a large collection of unreliable ones. This is uncomfortable to accept because more rules feel like better coverage, but the analyst who’s learned to ignore the noise in a high-volume SIEM is more dangerous than an analyst with a smaller, trustworthy signal.

flowchart LR
  A[Hunt or intel finding] --> B[Write Sigma rule in git]
  B --> C[Peer review / PR]
  C --> D[CI: convert + unit test]
  D --> E[Deploy to SIEM]
  E --> F[Validate: Atomic Red Team]
  F --> G{Fires cleanly?}
  G -->|Yes| H[Promote to production]
  G -->|No| B
  H --> I[Monitor FP rate + tune]
  I --> J{Still earning its keep?}
  J -->|No| K[Retire]

A note on coverage vs. fidelity

The natural progression is to want more coverage — a detection for every ATT&CK technique. Resist this. Coverage without fidelity is how you build a SIEM full of noise.

A better approach: start with a handful of high-signal, high-fidelity detections for behaviours that are almost never benign. Process injection, LSASS access, unusual parent-child process relationships, outbound connections from processes that have no business making them. Get those right and tuned before expanding. Each new rule you add is a maintenance burden — make sure it earns its keep.

The ATT&CK navigator is useful here as a visual gap analysis tool, but use it as a map, not a checklist. The goal is trustworthy coverage of the things that matter for your threat model, not coloured cells in a spreadsheet.

Next Up

Automation and SOAR: making your homelab respond, not just alert — when detections fire, what happens next shouldn’t depend on a human being awake.

Previous: Threat intelligence that isn’t just a feed

It isn’t. Intelligence is a process that produces decisions. The output isn’t a list of bad IP addresses — it’s an analyst saying “based on what I know about this adversary’s behaviour, we should be watching for X.” The feed is raw material at best. On its own, it’s noise with a confidence problem.

This distinction matters in a homelab context because it changes what you build and why. If you’re running MISP to ingest abuse.ch feeds and call it “threat intel,” you’ve built a fancy blocklist manager. Which is fine — but it’s not intelligence work. Intelligence starts when you start asking why and so what.

Let’s build that understanding from the ground up.

The Intelligence Lifecycle

The intelligence lifecycle is a six-phase loop. Every professional CTI program runs on it, whether or not they’ve named it that. At home you’re running a stripped-down version of the same thing.

Direction. Before you collect anything, you need to know what question you’re trying to answer. This is called the Priority Intelligence Requirement, or PIR, in formal programs. In homelab terms it sounds like: “I run self-hosted services on the internet. What threats are realistically targeting things like me?” That’s a PIR. It scopes everything downstream. Without it, you’re just downloading lists of malicious IPs and hoping.

Collection. Given your direction, you gather raw data from sources that might contain relevant information. Threat reports, feeds, open-source repositories, honeypot data, your own logs. Collection without direction produces data lakes. Collection against a specific question produces useful signal.

Processing. Raw data isn’t analysis-ready. You need to normalize formats (STIX/TAXII is a common standard, more on that in a moment), deduplicate, filter out things outside your scope, and structure it so a human — or a tool — can reason over it. This is often the least glamorous step and frequently the bottleneck. A lot of so-called “threat intel platforms” are really processing engines.

Analysis and Production. This is the actual intelligence work. You look at what you’ve collected and processed, find patterns, connect dots, and produce something a decision-maker can act on. “These indicators are associated with a phishing campaign targeting financial services orgs using this particular lure” is analysis. A list of 40,000 IPs is not.

Dissemination. Getting the product to whoever needs it. In a homelab this might be: updating your firewall rules, writing a hunt hypothesis to run in your SIEM, adding detection logic, or just noting in your notes file that a particular technique is active right now. The form follows the consumer.

Feedback. The loop closes here. Your consumers — which at home is mostly you — tell you what was useful, what wasn’t, and what questions remain unanswered. That feedback reshapes your next Direction cycle. This is the phase most people skip, and it’s why bad intel programs keep collecting things no one uses.

Strategic, Operational, Tactical

Not all intelligence serves the same purpose. The field divides it into three levels, and they’re consumed by different people for different reasons.

Strategic intelligence deals with long-term trends, geopolitics, and the “who and why” behind threats. Who are the major threat actors targeting this sector? What’s driving their motivation? What techniques are becoming more common over a multi-year timeframe? This level is consumed by executives and decision-makers. At home you’re engaging with strategic intel when you read a year-in-review threat landscape report and use it to decide whether to invest time hardening a particular service.

Operational intelligence sits in the middle. It describes active campaigns: how a specific adversary is operating right now, what infrastructure they’re using, what TTPs define their current activity. It answers “how are they doing it?” for a particular campaign. Analysts and threat hunters consume this level — it’s how you design hunts and detections that are relevant to what’s actually happening, not just theoretically possible.

Tactical intelligence is the atomic layer: indicators of compromise. Hashes, IP addresses, domains, file names, registry keys. Highly specific, highly actionable for automated systems, and highly perishable. An IP address that’s malicious today is often abandoned by next week. Your firewall and endpoint tools consume tactical intel.

The three levels form a pyramid in terms of durability. Tactical is the bottom — wide, abundant, and short-lived. Strategic is the apex — narrow, harder to produce, and it stays relevant for years.

IOCs vs TTPs — and Why It Matters

This connects directly to what we covered in Part 2 with the Pyramid of Pain. To recap: the Pyramid describes the cost an adversary pays when you detect and block at each level. Hash values at the bottom cost them nothing to change — one recompile. IP addresses, same story — spin up a new VPS. At the top of the pyramid are TTPs: the patterns of how they operate. Those are expensive to change because they’re baked into the adversary’s tooling, training, and habits.

The implication for intel work is the same: IOCs are perishable. A ThreatFox feed of malicious IPs is useful for blocking today’s infrastructure, but tomorrow that infrastructure may be gone and new IOCs will be different. A detection written against a TTP — say, PowerShell spawning from a parent process that shouldn’t spawn it — is durable. The adversary has to change their behavior, not just register a new domain.

This doesn’t mean IOC feeds are worthless. They block known-bad automatically and with low overhead. The mistake is treating them as intelligence rather than what they are: a low-level, high-volume, short-lived layer of defence.

If your intel work never rises above IOC collection, you’re defending against yesterday’s artifacts. The higher-value work is understanding the techniques in play and building detections that catch those techniques regardless of what infrastructure or tools the adversary uses.

The Diamond Model

When you’re analyzing a specific intrusion or campaign, the Diamond Model of Intrusion Analysis is a structured way to reason about what you know and what you can infer.

The model has four vertices:

Adversary — the threat actor. Who is conducting the activity? Their intent, motivation, and identity.

Capability — the tools, techniques, and malware they use. What they can do.

Infrastructure — the technical resources they use to operate: domains, IPs, C2 servers, email accounts. The “how they communicate with their tools” layer.

Victim — the target. Who is being attacked and what features make them a target.

The insight is that these vertices are connected — you can pivot from any one to learn about the others. You observe a piece of infrastructure (a C2 domain) and pivot to: what other domains use the same registrar and registration pattern? That might expand your infrastructure picture. You see a capability (a particular malware family) and pivot to: what adversaries are known to use this? That connects to the adversary vertex.

In practice, most homelab analysts are working from the infrastructure and capability vertices — those are what show up in open-source feeds. The Diamond Model gives you a mental scaffold for asking “what else can I learn from this?” rather than just adding an IOC to a list and moving on.

MITRE ATT&CK as the Shared Language

ATT&CK is the framework that connects intel work to detection engineering, which is Part 4. At its core it’s a structured catalog of adversary TTPs — organized by tactic (the goal: initial access, execution, persistence, etc.) and technique (the specific method: spearphishing attachment, PowerShell, scheduled task, etc.).

Its value in an intel context is that it gives you a common vocabulary. When a threat report says an adversary uses “T1059.001 — Command and Scripting Interpreter: PowerShell,” that’s a precise, standardized claim that you can immediately map to a detection engineering question: “Do I have coverage for that technique?” If you’re consuming intel from multiple sources in different formats, ATT&CK IDs give you a way to normalize and compare across them.

The practical workflow: when you read a threat report or process a feed that includes technique information, map it to ATT&CK. That mapping becomes the bridge to Part 4 — it turns intel into a detection requirement. “This campaign uses T1566.001 (spearphishing with an attachment) for initial access and T1059.001 (PowerShell) for execution” tells you exactly what techniques to build coverage for.

Free Sources Worth Your Time

You don’t need paid subscriptions to do meaningful intel work at home. These sources are realistic for a homelab.

A note on vendor reports: they exist partly for marketing. The Mandiant M-Trends report is genuinely useful; the accompanying press release is less so. Read the primary report, not the summary blog post. The data on dwell time, initial access vectors, and industry targeting distributions is valuable and grounded in real incident response cases.

Tools: MISP and OpenCTI

Two open-source platforms dominate the homelab intel space.

MISP (Malware Information Sharing Platform) is a mature, battle-tested platform for managing IOCs and sharing them across organizations. At home you’d use it to ingest feeds from abuse.ch, OTX, and others; tag and correlate events; and export IOCs to your firewall or SIEM. The MISP community runs sharing communities (called “MISP communities”) where orgs exchange intel — some are publicly joinable.

The honest assessment: MISP is powerful and comprehensive. It’s also complex to operate, with a lot of surface area for configuration. The learning curve is real. If your goal is structured IOC management and you’re willing to invest time, it pays off. If you just want to block feeds, a simpler option exists (your firewall can pull threat feeds directly).

OpenCTI is a newer platform built around a STIX2 knowledge graph. Where MISP is strong on IOC management and sharing, OpenCTI is stronger on the knowledge graph side — it’s designed to represent relationships between entities: “this adversary used this malware, which connects to this campaign, which targeted this sector.” It integrates natively with MITRE ATT&CK. It’s also heavier to run — it uses Elasticsearch and several dependent services and wants several gigabytes of RAM.

Neither is a weekend setup. Both have Docker Compose deployments that simplify the initial install, but plan for a week of configuration, connector setup, and getting feeds flowing before anything feels solid. The payoff is a platform that actually does intelligence work rather than just blocklist management — but be honest with yourself about whether you have the time to operate it properly.

A Worked Example: Feed to Hunt to Detection

This is the loop that makes intel useful rather than decorative.

Start on ThreatFox. Pull a recent submission — let’s say it’s a Cobalt Strike C2 domain with associated IP and a BEACON tag. The submitter has also provided a JA3 hash for the TLS fingerprint.

Process: You have three IOCs (domain, IP, JA3 hash) and a malware family (Cobalt Strike Beacon). You map what you know: Cobalt Strike is a commercial pen-test tool heavily abused by ransomware operators and espionage groups. You look up Cobalt Strike in ATT&CK — it maps across multiple techniques, but the relevant ones for Beacon are T1071.001 (Application Layer Protocol: Web Protocols, for C2 over HTTP/S) and T1095 (Non-Application Layer Protocol, for DNS beaconing).

Enrich: OTX may have a pulse on this indicator with additional context — what campaign, what targeting. The CISA KEV catalog may have related vulnerabilities if this Beacon deployment is associated with known exploitation activity.

Decide — hunt hypothesis or detection:

Hunt hypothesis route (loops to Part 2): “Do I have any traffic to this domain or IP in my logs from the past 30 days? Do I have any TLS connections matching this JA3 signature? Are there hosts on my network making high-frequency, low-payload HTTP connections consistent with Beacon’s beacon interval?” You run those queries in your log data.

Detection route (sets up Part 4): The JA3 hash and the behavioral pattern of Beacon — periodic callback intervals, relatively uniform payload sizes — can be turned into detection rules. The domain and IP can be added to blocklists with a note about why. More durably: if you know Beacon typically uses named-pipe communication for lateral movement, that’s a behavioral detection that survives this specific campaign moving to new infrastructure.

This is the handoff. Intel produces a hypothesis or a detection requirement. It doesn’t sit in a list. It becomes work.

What You’re Actually Building Toward

Intelligence without detection is academic. Detection without intelligence is blind. Part 4 is where these two disciplines connect properly: we’ll take the ATT&CK techniques that intel work surfaces and turn them into actual detection rules — Sigma, log queries, behavioral signatures — that run against real data.

The work you do here — mapping IOCs to techniques, understanding adversary behaviour at the TTP level, building the habit of asking “what does this mean for what I should detect” — is the foundation that makes Part 4 useful rather than a recipe you’re copying without understanding.

Next Up

Detection Engineering: From Intel to Rules — taking what you’ve learned about adversary behaviour and turning it into detections that actually run.

Previous: Threat hunting: hypotheses, not alerts

Threat hunting is the discipline of going looking before the alert fires. You assume something slipped through — because statistically, something always has — and you go look for evidence of it in your own telemetry. No alert required. You start with a hypothesis and go see if the data agrees.

That framing matters a lot. Threat hunting is not “running your SIEM harder.” It is not reviewing old alerts. It is proactive, hypothesis-driven search for threats that your existing detections missed. The moment you understand that distinction, the rest of this post makes sense.

Why bother hunting at home?

Fair question. You’re one person, running a homelab, protecting data that is probably not the target of an APT. The realistic threat model is automated botnets, opportunistic credential-stuffers, and the occasional cryptominer looking for an open door.

Here’s why hunting still makes sense:

First, it is the best hands-on education in this field. Reading about adversary behaviour is fine. Hunting through your own telemetry to look for it is a different category of learning entirely. The techniques are the same ones you’d use in a corporate SOC — the stakes are lower, which means you can experiment freely.

Second, every hunt you run produces one of two outputs: an incident you can investigate, or a new detection rule you can add to your monitoring stack. Even a hunt that finds nothing should produce a tested negative and a new detection idea. You’re feeding your detection engineering pipeline from post 04 of this series, whether you find something or not.

Third, if you’re building blue team skills for a career, “I hunt in my homelab” is a more credible answer than “I read about hunting” — and you’ll be able to talk concretely about what you found and what you built afterward.

The PEAK framework

PEAK was developed by the team at Splunk SURGe as a structured methodology for threat hunting. The name is an acronym for the three stages: Prepare, Execute, and Act with Knowledge. It gives you a repeatable loop rather than an ad hoc “I guess I’ll look at some logs” session.

Prepare

This is the upfront work that determines whether your hunt produces anything useful. In this stage you:

• Choose a hypothesis. A specific, falsifiable statement about adversary behaviour. Not “let me look for bad things” — that is not a hypothesis. Something like: “if an attacker used scheduled tasks for persistence, I’d expect to see new entries created around the time my telemetry shows unusual activity.”
• Scope the hunt. What time range? Which hosts? Which data sources? Narrowing scope keeps the hunt manageable and the results interpretable.
• Identify your data sources. Do you have the logs to test this hypothesis? If the answer is no, the Prepare stage has already been useful — you’ve identified a visibility gap.

The best hypotheses come from threat intelligence (what are real actors doing right now?) or from the MITRE ATT&CK framework (what techniques are used for this tactic, and do I have coverage?). More on ATT&CK in a moment.

Execute

You go get the data and analyze it. In this stage you:

• Gather and query the telemetry. Run the queries against your SIEM, log aggregator, or raw log files. Look for the artifact your hypothesis predicts.
• Analyze and pivot. If something looks interesting, pivot on it — look at the host in question across other data sources, expand the time window, chase related events.
• Keep notes. Every query you ran, every result that looked interesting or definitively normal, every dead end. This documentation is the output of the hunt whether you find something or not.

The goal in this stage is rigorous analysis, not confirmation bias. You are trying to falsify your hypothesis as much as confirm it.

Act with Knowledge

This stage is where the value gets captured and retained. In this stage you:

• Document findings. Positive or negative. What did you look for, what did you find, what did it mean?
• Create detections. If you found something, build a detection rule so you don’t have to hunt for it manually next time. If you found nothing but the hunt was valid, build the detection anyway — you’ve just proven the query works and can be automated.
• Hand off if needed. In a team context this means escalating to incident response or detection engineering. In a homelab context it means updating your monitoring stack and your runbook.
• Update your process. What data source did you wish you had? What hypothesis would you test next? The end of one hunt is the input to the next Prepare stage.

PEAK also defines three types of hunt you can execute within this loop:

• Hypothesis-driven hunts — start from a specific technique or behaviour you expect to find evidence of. Most structured hunts are this type.
• Baseline/exploratory hunts — look for deviations from what’s normal in your environment, without a specific hypothesis. Useful for discovering anomalies you didn’t know to look for.
• Model-assisted hunts (M-ATH) — use statistical or machine learning models to surface outliers for human analysis. This is the most sophisticated type, and in a homelab context it’s probably out of scope until you have strong baseline coverage first.

The Pyramid of Pain

David J. Bianco’s Pyramid of Pain frames threat indicators by how much it hurts an adversary when you detect and block them. It runs from the bottom — trivially easy for attackers to change — to the top, where evading detection requires re-engineering their entire operation.

The pyramid, bottom to top:

Most security tooling lives at the bottom of the pyramid. Hashes and IPs populate threat intel feeds. Blocking them is cheap for you to do and cheap for attackers to evade. You block a C2 IP, the attacker rotates to a new one in ten minutes.

The goal of threat hunting is to find indicators as high up the pyramid as you can. TTPs are behaviours — the way an adversary moves laterally, establishes persistence, exfiltrates data. These are harder to change because they reflect how the attacker thinks about compromising systems. If you can detect the behaviour rather than the specific tool or IP address, you’ve raised the cost of evading your detection enormously.

This is exactly where MITRE ATT&CK becomes useful.

MITRE ATT&CK as the hunt map

ATT&CK is a knowledge base of adversary tactics, techniques, and sub-techniques observed in real-world attacks. Think of it as a catalogue: for each thing an attacker might do — initial access, persistence, defense evasion, exfiltration — ATT&CK documents what it looks like, what data sources can detect it, and what mitigations exist.

For threat hunting, ATT&CK gives you a structured way to pick your next hypothesis. Instead of “I should look for suspicious things,” you can say “I haven’t tested my coverage of T1053 — Scheduled Task/Job. Let me build a hypothesis and hunt for it.”

A useful hypothesis format:

If [technique] happened in my environment, I would expect to see [specific artifact] in [specific data source], with [these characteristics].

That structure — if X happened, then Y would be observable — is what makes a hypothesis falsifiable. If you go look for Y and don’t find it, you’ve either confirmed X didn’t happen, or you’ve found a gap in your telemetry. Either outcome is useful.

Data sources you need at home

You can’t hunt what you can’t see. The minimum viable telemetry stack for a homelab hunt program:

Sysmon (Windows) — the Microsoft System Monitor installs as a driver and logs process creation, network connections, file system events, registry changes, and more with far more detail than Windows’ built-in event logging. Event ID 1 (process creation) and Event ID 3 (network connection) alone cover a large fraction of common hunting scenarios. If you run any Windows hosts in your lab, Sysmon is non-negotiable.

auditd + osquery (Linux) — auditd captures syscall-level activity on Linux hosts; osquery lets you query the OS state (running processes, network connections, loaded kernel modules) using SQL. Together they give you real-time and historical visibility into what’s happening on your Linux machines.

Zeek (network metadata) — Zeek sits on a network TAP or span port and generates structured logs: connection records, DNS queries, HTTP requests, SSL certificate metadata, and more. It does not capture full packet payloads (no PCAP bloat) — it produces analyst-friendly metadata. Even in a home network, Zeek on a Pi or small VM seeing your LAN traffic tells you things your endpoint logs cannot.

A worked hunt: beaconing via suspicious scheduled tasks

Let’s walk through a complete hunt end to end using the PEAK loop.

Hypothesis (Prepare):

If an attacker established persistence using a scheduled task (T1053.005 — Scheduled Task), I would expect to see Sysmon Event ID 1 (process creation) for schtasks.exe or at.exe with command-line arguments that create a new task, at a time or from a parent process that does not match my normal admin activity. I would also expect to see a corresponding Windows Task Scheduler event (Event ID 4698) in the Security log.

Data sources identified: Sysmon Event ID 1, Windows Security Event ID 4698.

Scope: All Windows hosts in the lab, last 30 days.

Execute — the query:

In a home setup you might be shipping logs to Elasticsearch, a Graylog instance, or even reading raw Sysmon .evtx files. Here is the logic in pseudo-SPL (Splunk-style) with comments that translate to any log query tool:

index=sysmon EventID=1                     # Process creation events only
  (Image="*\\schtasks.exe"                 # Direct schtasks invocation
   OR Image="*\\at.exe")                   # Legacy at.exe — rare but present in old TTPs
  CommandLine="*/create*"                  # Only care about task creation, not query/delete
| eval parent_suspicious=if(
    match(ParentImage, "(?i)(cmd\.exe|powershell\.exe|wscript\.exe|cscript\.exe)"),
    1, 0)                                  # Flag if parent is a shell or scripting host
| where parent_suspicious=1               # Drop routine admin activity (Task Scheduler service, etc.)
| table _time, ComputerName, User, CommandLine, ParentImage, ParentCommandLine
| sort -_time

Pair that with:

index=wineventlog source="WinEventLog:Security" EventCode=4698
# Event 4698 = "A scheduled task was created"
# Contains the task XML: shows trigger, action, author, run-as account
| table _time, ComputerName, TaskName, TaskContent, SubjectUserName
| sort -_time

Interpret the results:

If both queries return nothing: strong negative — no scheduled tasks were created from suspicious parent processes in this window. Document it and move on.

If Sysmon Event ID 1 shows powershell.exe spawning schtasks.exe /create /sc minute /mo 5 /tn "WindowsUpdate" /tr "C:\Users\Public\update.ps1" — that is a very loud indicator. Short interval, generic task name designed to blend in, script living in a user-writable path. Investigate that host immediately.

If you find legitimate admin activity that triggered the rule: tune the query to exclude your known-good paths and users, and document the exclusion. You’ve just made the detection more accurate.

Outcome and promotion:

If the hunt finds nothing suspicious, take the query logic and turn it into a persistent alert in your SIEM or log platform. You’ve validated that the query runs correctly and produces no false positives in your environment. Now it’s automated — you don’t need to run this hunt manually again unless your environment changes.

If it finds something: you have an incident. Work the incident, then build the detection once you understand the full scope of what happened.

That is the loop. Hunt → find or not-find → build detection → run next hunt.

The PEAK loop as a diagram

flowchart TD
  A[Hypothesis: technique X leaves artifact Y] --> B[Identify data sources]
  B --> C[Gather + query telemetry]
  C --> D{Evidence of Y?}
  D -->|Yes| E[Scope + investigate]
  E --> F[Respond + document]
  F --> G[Write a durable detection]
  D -->|No| H[Document the negative result]
  H --> I[Refine or retire hypothesis]
  G --> J[Hand to Detection Engineering]

Hunting up the Pyramid

One practical note on prioritizing your hunts: start at the bottom of the Pyramid of Pain and work up, but aim for the top.

Starting low (hashes, IPs) is easy and gives you quick wins, but those detections are fragile — an attacker can evade them trivially. Over time, push your hunts toward artifacts and TTPs. A hunt for “PowerShell spawned from a document viewer” is harder to build and tune than a hash blocklist, but it catches a whole class of attacks regardless of what specific payload was used. That’s the goal: detection rules that are robust against evasion because they target behaviour, not indicators.

ATT&CK maps every technique to observable artifacts in specific data sources. Use that mapping to work through your stack systematically. Sub-technique T1059.001 (PowerShell) → data source: Process creation, Command execution → look for Sysmon Event ID 1 with powershell.exe and suspicious command-line flags. T1078 (Valid Accounts) → Logon events → look for authentication from unusual source IPs or at unusual hours. Each hunt tells you something about your visibility and adds a layer to your detection coverage.

Quick reference: hunt planning checklist

Next Up

Post 03 covers threat intelligence at home — how to consume, filter, and operationalize external threat intel without drowning in noise: Threat Intelligence: Signal vs. Noise at Home.

Previous: Building a home SOC: the architecture

This post is that mental model. By the end you’ll know what a SOC is actually doing, what you need to collect, how the collection layer works, which SIEM is worth your time at home, and how to think about retention without starving your machine. Then we’ll get concrete.

The One Mental Model You Need

A SOC — Security Operations Center, corporate or homelab — is a single pipeline with six stages. Everything in this series maps to one of them:

Sources → Collection/Shipping → Parse/Normalize → Store/Index (SIEM) → Detect → Alert → Triage → Dashboards

That’s it. Strip away the vendor marketing, the product names, the conference theatre, and you have a pipeline that moves events from wherever they happened to a place where you can act on them. Each stage has exactly one job:

• Sources generate events: your Linux host, your firewall, your DNS server, your web server.
• Collection agents run on or near the source, read those events, and ship them somewhere central.
• Parsing and normalization translate “sshd[1234]: Failed password for root from 10.0.0.1 port 51234 ssh2” into a structured record with consistent field names — source IP, destination IP, event type, timestamp — so that detection rules can be written once and applied everywhere.
• The SIEM stores those normalized records and indexes them for fast search and correlation.
• Detection rules run against the event stream and produce alerts when something matches a pattern you care about.
• Alerting delivers those detections to you: a Telegram message, an email, a PagerDuty call if you’re running something enterprise-adjacent.
• Triage and dashboards are where a human looks at what the pipeline surfaced and decides whether it’s real.

Anchor everything on this. When someone tells you Wazuh is a SIEM, you now know enough to push back: it’s more accurately a HIDS with SIEM-adjacent capabilities that covers collection, normalization, and detection, with a dashboard that handles triage. When someone says Elastic is just a search engine, you can see that it’s the store/index layer that you wire the rest of the pipeline around. Names obscure. Stages clarify.

What to Collect at Home

The temptation is to collect everything. Resist it. You can only triage what you have time to look at, and a pipeline flooded with noise is worse than a narrower one you actually use. Collect what has signal. Add more when you’ve burned through the first layer.

Host logs — Linux

If you have Linux machines (and if you’re homelabbing, you do), start here. Two sources matter:

auditd watches system calls and produces a structured event for every file access, privilege escalation, process exec, and network connection that matches your rules. It’s noisy out of the box — you’ll need a ruleset like Florian Roth’s auditd rules to get meaningful signal without drowning in routine I/O. The payoff: it catches things like a process writing to /etc/passwd or a reverse shell spawning a bash child — the kind of activity that shows up in breach reports.

journald and syslog give you application logs, auth logs, and kernel messages. The most useful file is /var/log/auth.log (or journalctl -u ssh on systemd systems): every SSH login attempt, success, failure, and sudo invocation is there. Boring and essential.

Host logs — Windows

Windows Security Event Log is the primary source. The events that actually matter for detection: 4624 (successful logon), 4625 (failed logon), 4648 (logon with explicit credentials, common in pass-the-hash), 4688 (new process created — enable command-line logging), 4698/4702 (scheduled task created/modified), 7045 (new service installed). That’s your starter list; you don’t need all 1,000 event IDs.

Sysmon takes Windows logging from “adequate” to “genuinely useful.” It’s a Microsoft Sysinternals tool that runs as a driver and produces rich events for process creation, network connections, file creation, registry changes, and more — all with hashes and parent process info. The SwiftOnSecurity config is the standard baseline. If you have any Windows machines in your lab, install Sysmon on them. The improvement in detection quality is significant.

Network logs

Network visibility is what catches the things that never touch your hosts — lateral movement, command-and-control, DNS exfiltration.

Zeek (formerly Bro) runs on a tap or span port and produces structured logs for every protocol it understands: HTTP, DNS, TLS (metadata, not decrypted content), SSH, SMB. A single Zeek log line for a DNS query tells you the querying host, the query name, the response, the query type, and the answer TTL. That’s enough to catch DNS tunneling and C2 beaconing by pattern.

Suricata does signature-based detection on network traffic — it’s the IDS/IPS layer. Point it at your network interface and load the Emerging Threats ruleset and it will flag known-bad patterns: malware traffic signatures, exploit kit callbacks, cryptocurrency mining, anomalous protocols. False positive rate is manageable if you tune the rules to your network; untouched, it will generate a lot of noise you have to work through.

If you have a firewall — pfSense, OPNsense, or even just UFW logs forwarded from your hosts — collect it. Firewall logs tell you what connections were allowed and denied, which is the coarsest possible network visibility but still useful for spotting unexpected outbound connections or port scanning.

DNS — Pi-hole is a gift here

If you’re running Pi-hole for ad blocking (and if you’re homelabbing, you probably are), you’re already sitting on a DNS log source that most small organizations would pay for. Every DNS query from every device on your network passes through it. Enable query logging, ship those logs into your pipeline, and you get:

• A full record of every domain every device resolved
• Baseline normal DNS behavior per host (so anomalies stick out)
• Easy detection of devices talking to newly-registered domains, DGA patterns, or known-bad domains

It’s almost comically underused as a security telemetry source. Collect it.

Service logs

nginx access logs and error logs. SSH auth logs (already covered above). Any application logs your services produce. These are lower-priority than host and network, but they close the gap when you’re trying to reconstruct what happened to a specific service.

Collection and Shipping Agents

You have log sources. You need something to read them and move the events somewhere central. This is the collection layer, and the three tools worth knowing for a homelab are:

Wazuh agent — if you’re running Wazuh as your SIEM (more on this below), the Wazuh agent handles collection and ships to the Wazuh manager. It reads local log files, integrates with auditd, runs integrity checking, and handles a lot of the parsing. For a Wazuh-centric setup, you just deploy the agent and point it at the manager.

Elastic Agent / Beats family — if you’re running the Elastic Stack, Filebeat is the standard log shipper: it tails log files and ships them to Elasticsearch or Logstash. Winlogbeat is the Windows-specific equivalent for Event Log. Elastic Agent is the unified replacement for the individual Beats — it does collection, parsing, and detection in a single binary. More config overhead upfront, but cleaner at scale.

Fluent Bit — a lightweight, vendor-neutral log forwarder written in C. Lower resource footprint than the others, which matters when you’re running it as a sidecar on a machine that’s doing other things. Worth considering for resource-constrained hosts.

On normalization and why it matters

Raw log formats are a mess. sshd writes one thing, nginx writes another, Windows Event Log writes something completely different. Detection rules that operate on raw text are brittle — they break when a log format changes, and they can’t correlate across sources.

Normalization solves this by mapping everything to a common schema. The standard in the open-source world is ECS — Elastic Common Schema: consistent field names across event types (source.ip, destination.port, process.name, event.outcome). When your Linux auth log and your Windows Security log both normalize to ECS, you can write one detection rule that fires on failed authentication from any source. Without normalization, you write two rules, then six, then give up maintaining them.

The collection agent usually handles normalization before shipping. Wazuh has its own decoder/rule format. Elastic Agent includes ingest pipelines. The point is: don’t ship raw text if you can help it. Normalized events at rest mean faster detection rules and easier queries.

SIEM Choice for a Homelab

This is where most guides either pick a vendor without justifying it or give you a list so long it’s useless. Here’s an honest comparison:

My concrete recommendation for starting out: Wazuh.

Here’s why. Security Onion is technically impressive but it’s a serious machine commitment and the setup complexity front-loads a lot of friction that gets in the way of actually learning detection. Elastic is fantastic but you’ll spend your first month configuring pipelines before you get a single useful alert. Graylog is strong but leans toward log management rather than security operations.

Wazuh hits the practical sweet spot: real HIDS capability out of the box, a detection rule library that covers common attack techniques, a web dashboard for triage, and enough documentation that you can troubleshoot when things go sideways. Once you’ve outgrown Wazuh — once you’re shipping network logs and want custom correlation rules and a graph-based threat hunt view — you’ll know exactly what you need and why. Start there.

T1/T2/T3 Reframed for a Solo Homelabber

In enterprise security, T1/T2/T3 refers to analyst tiers — different people with different skill levels handling different depths of investigation. In a homelab, you are all three tiers, and that’s actually fine. Reframe the tiers as workflow stages you rotate through:

T1 — Triage: An alert fired. Is it real? Is it a known false positive? Is this something that can be closed in under two minutes? This is where you spend the most time. The goal is to close the noise as fast as possible so the signal can surface. Good detection rules and a well-tuned baseline cut T1 time dramatically — which is why normalization and rule quality matter more than alert volume.

T2 — Investigate: Something passed T1 and looks real. Now you’re correlating: what else was happening on that host at that time? Was there network activity to match? What processes were running? This is where good logs pay off. If you only collected auth logs, you’re guessing. If you have auditd, Sysmon, and Zeek, you can reconstruct the timeline.

T3 — Hunt and engineer: No alert fired, but you want to look for something specific. Or you found a gap in your detection coverage and want to write a rule for it. This is the proactive mode: threat hunting, detection engineering, building the playbooks that make T1 faster next time.

A healthy homelab SOC practice rotates through all three. Don’t just watch the alert feed — spend time at T3 even when nothing is firing.

Storage and Retention Reality

This is where homelab SOCs usually go wrong. You set up a SIEM, it starts ingesting logs, and three months later your disk is 90% full and Elasticsearch is refusing writes.

The practical framework: hot vs. warm vs. cold.

• Hot data is fully indexed, searchable in seconds, in active retention. This is your recent events — the last 30 to 90 days, depending on your disk.
• Warm data is compressed, slower to search, but still accessible. Some stacks support this natively (Elasticsearch ILM, for example).
• Cold is archived off the box entirely — a tarball on a NAS, a compressed export on an external drive. You need this for incidents where the relevant activity happened months ago.

What’s a sane retention window at home? For hot data: 30 days is workable on most single-box setups. 90 days if you have the disk. For security research rather than incident response, even 14 days gives you enough context for most detections.

Rough disk math: Wazuh with a few agents and moderate log volume runs somewhere between 5–20 GB per week depending on how much you’re collecting. Zeek on a busy home network can add another 5–15 GB per week. Plan from there — don’t add sources faster than you add disk.

Quick Reference: The Full Pipeline

Next Up

With the architecture clear, the next post gets into the work that actually makes a SOC useful: Threat Hunting in Your Homelab — proactive detection, hypothesis-driven hunting, and how to find things that didn’t fire an alert.

Previous: Why run a SOC at home

For a homelab of one, the scope collapses considerably. You’re not staffing shifts. You’re not managing a SLA for mean-time-to-detect. You’re building the same mental models and the same muscle memory that blue team practitioners use at work — just on infrastructure you own, with telemetry from services you control, in a lab where breaking things costs you nothing but time. That’s the value. The reps are real even when the stakes are low.

This is the opening post for a new series. The beginner series got you a hardened VM and a working homelab. This series gets you a security practice built on top of it.

What “SOC” actually means when it’s just you

In an enterprise, a SOC is a people-process-technology operation: analysts triaging alerts, incident responders jumping on confirmed threats, engineers tuning detections, intelligence teams tracking adversaries, and a SIEM stitching the telemetry together. The technology is expensive. The headcount is substantial.

At home, you have none of that. What you have is:

• Technology: the same open-source tools the industry builds on — Elasticsearch or Loki for log aggregation, Grafana or Kibana for visualization, Zeek or Suricata for network visibility, Sigma rules for detection logic
• Process: a habit of actually looking at your logs, and a consistent way of recording what you find
• People: you, wearing all the hats

The hat-switching is actually useful. When you’re writing a detection, you’re doing detection engineering. When you’re looking for something that hasn’t triggered an alert yet, you’re threat hunting. When you’re figuring out what a piece of malware does so you know what to look for, you’re doing threat intelligence. Doing all of these yourself — even at small scale — gives you a systems view of how the disciplines depend on each other that’s hard to get any other way.

Why bother

The honest answer is that it builds real skills on real telemetry.

When you tune a Sigma rule against your own logs and it produces three false positives, you understand false positives in a way you don’t when it’s someone else’s lab data. When you catch a misconfigured service broadcasting on a port it shouldn’t be, you’ve just done your first detection. When you trace an authentication failure back to a device you forgot was still configured to talk to a service you decommissioned, you’ve done your first incident investigation. These are not toy exercises. The data is live, the infrastructure is real, and the analysis is yours.

You also genuinely will find things. Not nation-state intrusions — that’s not the threat you’re defending against. But commodity nastiness is everywhere. Bots scanning for exposed services, credential stuffing attempts on anything you’ve accidentally left internet-accessible, malware on a device on your network that you didn’t know was there. A security practice that’s functional — even a simple one — catches that class of problem. The reps happen to have real payoffs.

The five disciplines this series covers

These aren’t five separate fields. They’re a loop. Each one feeds the others, and the loop is what makes a security practice work over time rather than just being a pile of tools.

SOC — collect and watch. This is the foundation: getting logs from your services and devices into one place, and having dashboards and alerts that surface anomalies. Without this layer, everything else is guessing. We’ll build a lightweight but functional log aggregation setup that gives you actual visibility into what’s happening on your network.

Threat Hunting — proactively look with no alert. Alerts catch what your detections are tuned to find. Threat hunting is the discipline of going looking for things that haven’t triggered anything yet — following a hypothesis (“do I have any hosts talking to IP ranges I don’t recognize?”) and investigating until you have an answer. It’s the proactive complement to reactive alerting, and it’s where experienced practitioners find the stuff that slips through.

Threat Intelligence — know what to look for. Intelligence is context: who is attacking what, with which techniques, and leaving which artifacts. At the homelab level, this means consuming free intelligence feeds (abuse.ch, MISP communities, CISA KEV) and using that context to prioritize what you’re watching for. It’s the input that makes detection engineering less of a guessing game.

Detection Engineering — turn knowledge into durable, tested detections. Once you know what to look for, you write detections — rules that fire when that thing shows up. The “engineering” part matters: detections should be tested against known-good and known-bad data, versioned, and maintained like code. A detection that produces too many false positives gets ignored. One that’s never been tested against a real example of the threat it claims to catch isn’t a detection, it’s a wish.

Automation / SOAR — let machines do the boring response. Security Orchestration, Automation, and Response sounds fancy. At home it means: when an alert fires, the obvious first response steps happen automatically, and the alert you receive already has the context you need to decide what to do next. Enrichment lookups, automatic isolation of known-bad IPs, ticket creation, Telegram notifications with the relevant log snippet attached — all of this is automatable, and automating it means you spend your time on decisions, not on running the same lookup for the fifth time.

The loop looks like this: telemetry from your SOC layer informs threat hunting; hunting discoveries become intelligence; intelligence shapes detection engineering; detections feed back into the SOC layer; automation handles the response so you’re only interrupted when human judgment is actually required. Run the loop long enough and your practice gets tighter with every iteration.

What you need to follow along

You need a working, hardened homelab VM. That’s it for prerequisites.

If you don’t have one yet, the beginner series covers it: Getting the VM ready and Securing the Box get you to a solid baseline. If you’ve already been through that series, the starter projects post is a useful on-ramp — specifically Project 5, monitoring and alerts. Having even a minimal log aggregation setup already running means the first post in this series has something to build on.

The tools we’ll use are open-source and chosen because they’re what practitioners actually use — not because they’re the easiest to install or the most beginner-friendly. There’s a learning curve in places. That’s part of the point.

Next Up

The first technical post: SOC Architecture for a Homelab of One — what to collect, where to store it, and how to build a logging pipeline that’s actually useful without eating your weekend every time you need to maintain it.

Previous: Starter projects worth your first weekend

This post is the stuff.

I’ve picked five projects in rough order of difficulty. None requires a CS degree. All of them teach you something that compounds — each rung makes the next one easier. Do them in order if you want a curriculum, or jump to whichever itch you need to scratch first.

Project 1 — Run a local model with Ollama and a chat UI

What it is: Ollama is a tool that downloads and runs open-weight language models locally — Llama, Mistral, Gemma, and a dozen others — and exposes them over a simple API. Pair it with Open WebUI (a browser-based chat interface) and you have a ChatGPT-alike running entirely on your own hardware, sending nothing to anyone.

What you’ll learn: This is post 01’s theory made tangible. You’ll feel the difference between a 7B and a 13B model when your machine slows to a crawl loading the bigger one. You’ll hit a context window limit mid-conversation and understand why that number matters. You’ll discover that a quantized 4-bit model runs fine on 8 GB of RAM and sounds surprisingly good. None of this sticks from reading — it sticks from watching a progress bar crawl and a model finally answering you.

Rough effort: An evening. Ollama installs in one command. Open WebUI ships as a Docker image. If your reverse proxy is already working from earlier posts, you’ll have it behind a subdomain in under two hours. The only wildcard is download time — some models are several gigabytes.

Why it’s a good rung: It closes the loop on everything abstract from the first post. And once the API is running locally, every other project on this list can use it as a backend — you’re building infrastructure, not just a toy.

One thing to watch: Start with a small model (3B or 7B). It’s tempting to grab the biggest model that fits on your machine, but smaller models answer faster and the quality difference matters less than you expect for simple tasks. Get comfortable with the workflow first.

Project 2 — Self-host a useful service you’d actually use daily

What it is: Pick a service that replaces something you already use — a bookmarking app, a read-later list, a personal homepage that shows you your calendar, the weather, and quick links. Linkding is a clean, minimal bookmark manager. A homepage dashboard (several options exist in the self-hosting community) lets you wire in status widgets for your other services.

The specific tool matters less than the pattern: you’re running a container, pointing your reverse proxy at it, and ending up with a real URL you visit on purpose.

What you’ll learn: Docker Compose in practice — environment variables, volume mounts, the difference between a container restarting itself and a container that stays dead. Basic reverse proxy routing: how a request for bookmarks.yourdomain.local becomes a request to localhost:8765 on your server. And crucially, you’ll learn what it feels like to run something that matters to you, which changes how carefully you treat it.

Rough effort: Easy to medium, depending on the service. Most popular self-hosted tools have a docker-compose.yml in their README. Budget an evening for setup, and another hour later when you inevitably forget what you did and have to read your own notes (you did take notes, right — more on that below).

Why it’s a good rung: This one isn’t AI. That’s the point. Homelab is broader than AI, and the Docker + reverse proxy muscle memory you build here is the exact same muscle you use for every AI service later. It’s also a low-stakes environment to break things — if your bookmark manager goes down for a day, the consequences are mild.

Project 3 — A “chat with your docs” setup (RAG)

What it is: Retrieval-Augmented Generation, or RAG, is the pattern where you load your own documents — notes, PDFs, articles — into a vector database, and the AI searches that database before answering. Instead of the model hallucinating from its training data, it reads your actual content first.

Tools like Anything LLM or Open WebUI’s built-in document mode let you drop in a folder of PDFs and start asking questions within an hour, no code required. If you want to understand what’s happening under the hood, there are Python-based tutorials that walk you through building the retrieval layer yourself.

What you’ll learn: Why embeddings exist — your documents become numerical vectors, and “similarity search” means finding vectors near yours. Why context windows matter in practice: you can’t stuff 500 pages into a single prompt, so the retrieval step selects the relevant chunks. What chunking strategy means and why it affects answer quality. This is where the theory of post 01 stops being abstract.

Rough effort: A weekend, realistically. The tooling is better than it was two years ago, but you’ll spend time on document formatting, chunk size tuning, and being surprised when the model ignores a document that you’re certain contains the answer. Budget time to experiment.

Why it’s a good rung: RAG is the most common pattern in real enterprise AI systems. Understanding it from first principles — not just clicking “upload PDF” in a SaaS tool — is a durable skill. And the moment you get a good answer from your own notes, it clicks why people care about this.

Project 4 — A simple agent that does one useful job on a schedule

What it is: An “agent” in the practical sense: a script that uses a model (local or API-based) plus one or two tools to do a task automatically. The canonical beginner version is a morning briefing — fetch your RSS feeds, summarize the ten most recent items, send yourself a Telegram or email with the digest. Another version: watch a folder for new files and summarize or tag them.

You can build this in Python with a cron job and 50-100 lines of code. If you’re not a Python person, n8n (a self-hosted workflow automation tool) lets you wire this together visually.

What you’ll learn: Tool use — what it means for an AI to “call a function” rather than just generate text. The mechanics of a cron job and why scheduled automation is surprisingly hard to get right (time zones, error handling, the job that silently fails for three weeks). Prompt engineering for a specific task: how you instruct the model to summarize vs. extract vs. classify, and why the instructions matter a lot.

Rough effort: Medium to ambitious. The happy path takes an afternoon. Getting it robust — so it handles an RSS feed that goes down, or a model response that’s malformed, or a cron job that fires twice — takes longer. Start simple and add resilience later.

Why it’s a good rung: This is where “I have a homelab” becomes “my homelab does things for me.” It’s also the point where you start thinking about reliability and failure modes, which is the shift from hobbyist to practitioner. Post 01’s section on agents was abstract. After this project it isn’t.

Project 5 — Wire up monitoring and alerts for something you run

What it is: Pick one of your running services and instrument it properly. Uptime Kuma is a self-hosted uptime monitor — you point it at your services’ URLs and it pings them on a schedule, then notifies you (Telegram, email, ntfy, or others) when something goes down. That’s the floor. The ceiling is a proper metrics stack — Prometheus, Grafana, and node-exporter — which gives you CPU, memory, and disk graphs over time.

Start with Uptime Kuma. Add a metrics stack when you’re curious why something is slow.

What you’ll learn: What “observability” means in practice: the difference between knowing something is down (uptime monitoring) vs. knowing why it’s slow (metrics). How to configure alerting that’s useful without being spammy — alert on what needs human attention, not everything that twitches. And the uncomfortable lesson that services fail in ways you didn’t anticipate, and you only know because you were watching.

Rough effort: Uptime Kuma is a single Docker container and fifteen minutes of configuration. The full metrics stack is a weekend. Start with the easy version and let curiosity pull you further.

Why it’s a good rung: This teaches “production thinking” — the mindset shift where you’re not just running services but running services reliably. Every serious homelab eventually gets a monitoring layer, and the people who add it after their second mystery outage wish they’d added it after their first. If you’re building anything others depend on, this isn’t optional.

General advice before you start building

Start smaller than you think you need to. The temptation is to plan the full system — monitoring, agents, RAG, the works — and build it all at once. This leads to a half-assembled pile that lives in Docker Compose limbo for weeks. Pick one project. Finish it. Then pick the next.

Snapshot before you tinker. This was in post 03 and it’s worth repeating here because you’re about to actually need it. Before you upgrade a container, change a config, or “quickly try something,” take a snapshot. The five seconds this costs is the best insurance you have. The restore you do at 11 PM on a Tuesday will be the fastest you’ve ever appreciated a past decision.

Keep a notes file. Sounds obvious. Almost nobody does it until they’ve forgotten what they did to fix something the first time and spent two hours redoing it. A plain text file in your home directory, a note in Obsidian, a private GitHub repo — the format doesn’t matter. Write down what you installed, what config you changed, and why it didn’t work before the thing that made it work. Future you will be embarrassingly grateful.

Don’t expose anything to the internet without thinking. Post 04 covered this, but it bears a callback here: the projects above are designed to live on your local network, behind your reverse proxy, accessible only to you. The moment you poke a hole in your firewall or set up an external tunnel, you’ve changed the threat model. That’s sometimes fine and sometimes not — but it’s a decision to make deliberately, not accidentally.

That’s the arc

Six posts ago — post 00 — you had a laptop and a vague idea about AI homelabs. Since then you’ve covered how language models actually work, chosen hardware, built a VM, locked it down, and now you have a ladder of real projects to climb.

That’s not nothing. Most people who say they want to run local AI never get past “I should look into that.”

The homelab mindset is: ship something small, break it, fix it, understand it better than before. Repeat. The projects above are designed to be rungs, not destinations. After project 5 you’ll be ready to ask your own questions — “what if I wired the RAG setup into the agent pipeline?” — and you’ll have the vocabulary and the infrastructure to find out.

The best thing about running your own stack is that no one’s rate-limiting you, no one’s reading your prompts, and if something breaks you get to be the one who fixes it. That’s the whole deal. Keep climbing.

Previous post: Securing the Box — Series start: Start Here

We’re going to harden SSH properly — it’s cheap, correct, and you should just do it — and then I’m going to tell you the uncomfortable thing that most tutorials skip: none of that matters if someone can walk up and touch your machine.

Let’s go in order.

Part A — SSH Hardening

Your VM from post 03 is running. Someone can SSH into it with a username and password. That’s fine for a first boot. It stops being fine the moment anything on that machine is accessible from outside your LAN, and even on your LAN it’s sloppy hygiene. Fix it now, before you build anything on top.

1. Key-Based Authentication

Passwords can be guessed, leaked, or phished. SSH keys cannot be guessed — they’re mathematically intractable. This is the single highest-leverage change you’ll make.

On your local machine (not the server):

# Generate an ed25519 keypair — modern, fast, short keys
ssh-keygen -t ed25519 -C "your-label-here"
# Accept the default path (~/.ssh/id_ed25519) or name it something meaningful
# Set a passphrase — this protects the key if your local machine is compromised

Copy the public key to your server:

ssh-copy-id your-user@labvm
# Prompts for your password one last time, then installs the key

Test it immediately in the same terminal:

ssh your-user@labvm
# Should log in without asking for a password

If it works, keep this session open. You’ll need it in the next step.

2. Disable Password Auth and Root Login

Open /etc/ssh/sshd_config on the server:

sudo nano /etc/ssh/sshd_config

Find and set these lines (uncomment them if they’re commented out):

PasswordAuthentication no
PermitRootLogin no
PubkeyAuthentication yes

Before you restart sshd, open a second SSH session in a new terminal. If you misconfigured something and lock yourself out, you want an existing session to fix it.

# In a NEW terminal window/tab, confirm you can still log in with your key:
ssh your-user@labvm
# If this works, you're safe. If it doesn't, don't restart sshd yet — go fix the config.

Once confirmed:

sudo systemctl restart ssh

Try a third connection from scratch. No password prompt means it worked.

3. Limit Exposure: Port and AllowUsers

Change the default port. This is not real security — anyone running a port scan finds it in seconds. It’s friction. It cuts about 95% of the automated noise from bots that only hammer port 22, which keeps your logs readable. Call it what it is.

In /etc/ssh/sshd_config:

Port 2222

Pick any unprivileged port (1024–65535) that nothing else is using. Update your client’s ~/.ssh/config to match:

Host labvm
    HostName 10.0.0.50
    User your-user
    Port 2222
    IdentityFile ~/.ssh/id_ed25519

Restrict who can log in with AllowUsers. Even if a user account exists on the server, only listed users can authenticate via SSH:

AllowUsers your-user

Restart sshd after any change. Always test from a second session first.

4. Firewall with ufw

A firewall that defaults to allowing everything is a firewall that isn’t doing anything. Set the policy to deny everything, then punch only the holes you need.

sudo ufw default deny incoming
sudo ufw default allow outgoing

# Allow SSH on whatever port you chose
sudo ufw allow 2222/tcp

# Allow any other services you're actually running and need access to
# sudo ufw allow 8080/tcp   ← example: a web UI you want reachable on LAN

sudo ufw enable
sudo ufw status verbose

Verify your SSH port is listed as ALLOW before you close your existing session. If you forget to allow SSH and enable ufw, you will lock yourself out. (The fix is to access the machine physically or via the hypervisor console — which is a preview of Part B.)

5. fail2ban

fail2ban watches your logs and temporarily bans IPs that fail authentication too many times. It won’t stop a targeted attack, but it stops the automated credential-stuffing bots that run continuously across the entire internet.

sudo apt install fail2ban -y
sudo systemctl enable --now fail2ban

The defaults are reasonable for SSH. If you want to tune the ban threshold:

sudo nano /etc/fail2ban/jail.local

[sshd]
enabled = true
port    = 2222
maxretry = 5
bantime  = 3600
findtime = 600

maxretry = 5 means five failures within ten minutes (findtime) earns a one-hour ban (bantime). Adjust to taste. Check current bans:

sudo fail2ban-client status sshd

6. Keep It Off the Public Internet

This is worth more than all five steps above combined.

Do not port-forward SSH from your home router. Do not expose any homelab service directly to the internet unless you have a very specific reason and know exactly what you’re doing. The moment your SSH port is internet-routable, you are fighting every automated scanner and credential bot on the planet. fail2ban slows them down. A VPN makes them irrelevant.

Use a mesh VPN — Tailscale is the easiest entry point, WireGuard if you want to run your own. The idea: your homelab machine and your laptop are connected through an encrypted overlay network. From the public internet’s perspective, your SSH port doesn’t exist. From your laptop’s perspective, you SSH to a private mesh address as if you were on the same LAN.

Tailscale’s free tier handles most homelabs comfortably. Setup takes about ten minutes:

# On the server
curl -fsSL https://tailscale.com/install.sh | sh
sudo tailscale up

# On your local machine — install the Tailscale client for your OS,
# then authenticate both devices to the same account.
# They'll get addresses in the 100.x.x.x range.

# SSH via the mesh address — no router port-forwarding needed
ssh your-user@100.x.x.x

Once this is working, there’s no reason to ever expose SSH to the open internet. The attack surface collapses to: people who can authenticate to your Tailscale account. That’s a much smaller problem.

Part B — The Physical-Access Truth

Now for the part that most SSH hardening guides bury in a footnote or skip entirely.

Everything you just did — key auth, disabled passwords, ufw, fail2ban, mesh VPN — protects you against network-based attacks. An attacker trying to reach your machine over the wire hits a wall.

An attacker who can physically touch your machine doesn’t care about any of it.

What Physical Access Actually Lets Someone Do

Here’s the concrete attack, step by step, so it’s not abstract:

1. Boot from a USB drive (any live Linux distro)
2. Mount your server’s disk — it appears as a regular volume
3. chroot into it, run passwd your-user, pick a new password
4. Reboot normally, log in with the new password
5. They now have full access with your account

Alternatively: pull the drive, put it in a USB enclosure, plug it into their own machine. Read everything. Copy everything. Walk out.

This is not an exotic technique. It requires no special knowledge. It’s in every security curriculum as “basic physical access attack.”

The Actual Mitigations

Full-Disk Encryption (LUKS)

This is the real answer. If the disk is encrypted, pulling the drive or booting from USB gets an attacker a pile of ciphertext. Without the passphrase or key, it’s useless.

On Ubuntu/Debian, LUKS encryption is offered during OS installation. Enable it. The cost is: you have to enter a passphrase each time the machine boots (or configure a key file, which has its own tradeoffs). For a homelab VM that doesn’t reboot often, this is a minor inconvenience with meaningful protection.

If you didn’t encrypt at install time, there’s no painless retrofit — you’d need to reinstall. Worth knowing for your next build.

# Check if your current install is LUKS-encrypted:
lsblk -o NAME,FSTYPE,MOUNTPOINT | grep crypto
# or
sudo dmsetup ls --target crypt

If you see crypto_LUKS in the output, you’re already encrypted. If not, file it away as “do this on the next machine.”

BIOS/UEFI Password and Boot Order

Set a BIOS/UEFI password and configure the machine to boot only from its primary disk. This slows down the USB-boot attack — an attacker has to clear CMOS or disassemble the machine to get past it. Not impossible, but it adds friction and time.

On most motherboards this is in the UEFI settings under “Security” or “Boot.” The exact menu varies by vendor. Look for “Supervisor Password” and “Boot Priority.”

This is a speed bump, not a wall. Someone with enough time and physical access can bypass it. Combined with LUKS, though, you’re raising the effort level considerably.

Where the Box Physically Lives

For a home setup, this is underrated. A server in a locked room is more secure than an internet-hardened machine sitting on an open desk in a common area. Locks and doors are physical security controls, and they work.

You don’t need a cage or a data center. A spare room with a lock, a closet, an office you control — all of these reduce your exposure to the realistic threat model (roommate curiosity, visitors, opportunistic theft) significantly. A door lock costs zero dollars and takes no configuration.

The Honest Takeaway

Harden SSH because it’s cheap, correct, and good hygiene. You should do it regardless. But be honest with yourself about what it protects: your network perimeter, not your hardware.

If you’re running a homelab with data you actually care about — credentials, personal files, anything sensitive — full-disk encryption on anything that could leave your physical control is the right call. The box in your home office is probably fine sitting behind a locked door. The laptop you carry around, the external drive with backups, the NUC you’d take to a friend’s place: encrypt those.

And if someone ever breaks in and takes the hardware, the disk encryption means they got hardware, not data. That’s a bad day, but it’s a recoverable one.

Quick Reference: What Each Layer Buys You

The pattern is obvious. The software controls are all good and worth doing. Physical is where the game actually ends.

Next Up

Post 05 gets back to building things: Starter projects worth your first weekend — a short list of homelab projects that are genuinely useful day one, not just impressive on paper. Now that the box is hardened, we can actually put it to work.

Previous: Getting the VM ready

The short version: you want a virtual machine. Everything else flows from that decision.

Why a VM and not bare metal

The honest answer is reversibility. When you install things directly onto your laptop or a spare PC and something breaks, untangling it is tedious. When you do it inside a VM, you delete the VM and start over. That’s the whole pitch.

More concretely:

• Snapshots. You can freeze the state of a VM before doing something risky and roll back to that exact point if it goes sideways. This is not a feature most operating systems expose natively. It’s a superpower.
• Isolation. Your AI services don’t share libraries or ports with your daily-driver environment. You can install conflicting versions of Python, Node, whatever, without caring.
• Reproducibility. If you ever want to rebuild the lab from scratch on better hardware, the process is just “spin up a new VM, follow the same steps.” You’re not reverse-engineering a years-old hand-rolled setup.

The mental model I find useful: your homelab should be disposable and reproducible. The VM is disposable. The knowledge and configs you build up are what you keep.

Where to actually run the VM

You have three realistic options. None is universally correct.

Option A: Your existing laptop or desktop, with VirtualBox (or similar)

VirtualBox is free, runs on Windows, macOS, and Linux, and is the lowest-friction way to start. Download a Linux ISO, create a VM, you’re done in 20 minutes.

The catch: your laptop sleeps. Your homelab sleeps with it. If you just want to learn and experiment and don’t care about services being up overnight, this is fine. If you want a Telegram bot or a scheduled scraper running at 2 AM, this will frustrate you quickly.

Use this option if you’re not sure yet whether you want a homelab, or if you genuinely don’t have other hardware.

Option B: A spare PC or mini-PC running Proxmox (recommended)

This is the sweet spot. A mini-PC — the kind of small box that costs $150-300 used — draws 10-15W at idle, sits on a shelf, and runs 24/7 without complaint. You install Proxmox VE on it (type-1 hypervisor, runs directly on the hardware), then spin up one or more Linux VMs on top.

Why Proxmox specifically: it’s free, mature, well-documented, and has a solid web UI for managing VMs without touching a command line. Snapshots, backups, and VM cloning are first-class features. It’s also what most homelab guides assume, so you’ll find help easily.

If you’re in this for the long run and you have any spare hardware, this is where to land.

Option C: A cloud VPS

If you have no spare hardware and don’t want to buy any, a cheap VPS (a small virtual private server on a provider like Hetzner, DigitalOcean, or Vultr) gets you a 24/7 Linux box for $5-10/month. You can run everything in this series on it.

The trade-offs: it’s not “at home,” so it’s less private. You’re paying a recurring cost. And you can’t do GPU passthrough later if you want to run local LLMs. It’s a valid fallback, not the ideal destination.

Provisioning a base Linux VM

For the rest of this series I’ll assume Ubuntu Server LTS or Debian. Either works. Both are stable, well-supported, and have packages for everything you’ll need.

Step 1: Get the ISO. Download the latest Ubuntu Server LTS from ubuntu.com, or Debian from debian.org. The server edition has no desktop environment — that’s what you want. Less overhead, fewer attack surface.

Step 2: Create the VM. In Proxmox, click “Create VM” and walk through the wizard. In VirtualBox, click “New.” The wizard will ask you for:

• Name: something you’ll recognize. labvm or homelab is fine.
• Type/OS: Linux, Ubuntu or Debian as appropriate.
• Disk, RAM, CPU: see the callout below.
• Network: NAT works for VirtualBox. In Proxmox, use a bridged adapter so the VM gets its own IP on your LAN — this matters when you want to reach services from your phone or other devices.

Step 3: Install the OS. Attach the ISO, boot the VM, follow the installer. When it asks about additional packages during setup, skip them — you’ll install what you need. The one thing to select during install: OpenSSH server. You want SSH access from day one.

First-boot setup

Fresh install, logged in, cursor blinking. Here’s the checklist I run on every new VM before I do anything else:

# 1. Update everything
sudo apt update && sudo apt upgrade -y

# 2. Set your hostname (pick something meaningful to you)
sudo hostnamectl set-hostname labvm

# 3. Set your timezone
sudo timedatectl set-timezone America/Toronto
# Check available zones: timedatectl list-timezones | grep America

# 4. Create a non-root sudo user (if the installer didn't already)
sudo adduser yourname
sudo usermod -aG sudo yourname

# 5. Install essentials
sudo apt install -y curl git unzip htop

# 6. Confirm SSH is running (it should be from the install step)
sudo systemctl status ssh

That’s it. A few minutes of work. The VM is now in a clean, known state with a real user account and the basics installed.

One thing worth emphasizing: don’t do your regular work as root. The installer creates a root account; you should use a non-root user with sudo for everything day-to-day. It’s a habit that prevents a surprising number of accidents.

Take a snapshot right now

Before you install a single service or run another command, take a snapshot.

In Proxmox: select the VM, go to Snapshots, click “Take Snapshot.” Give it a name like clean-install. Done.

In VirtualBox: with the VM powered off, go to Machine > Tools > Snapshots, click “Take.” Same idea.

Why now, before anything is installed? Because this is the cleanest state the VM will ever be in. If anything goes wrong in the next post, the next month, or the next year, you can roll back to this snapshot and get a fresh start in seconds without downloading ISOs or running through the installer again.

I take a snapshot before any significant change: before upgrading a major package, before trying a new configuration, before installing something unfamiliar. It costs almost nothing and has saved me from painful rollbacks more times than I can count. Make it a reflex.

What’s next

You have a running VM, a non-root user, and a clean-install snapshot. The VM is reachable over your local network — but right now, anyone on that network (or anyone who can reach it) could potentially poke at it.

Before you install anything useful, you want to lock the front door. The next post covers securing this VM: setting up SSH key authentication, disabling password login, and understanding why SSH is how you’ll actually drive this machine day-to-day (not the hypervisor console).

Next up: Securing the box: SSH & the physical-access truth

Previous: Choosing your stack