Incident notes and pitfalls from a one-person homelab — a security engineer's actual infrastructure, written up as it breaks and gets fixed. No vendor pitch, no tutorial fluff.
Run by one operator, no teamStack Debian · systemd · Suricata · FastAPIAds / trackingnone
iOScampus, 12,000 people
Windowsregional office
NOXone desk, one lamp
Field Report
Your AI coding agent trusts your vendor's website more than it should
Not one of our own incidents this time — third-party research, worth flagging because it touches the exact toolchain this journal is written with.
Google recommends that companies publish an llms.txt file at their domain root — a plain-text, machine-readable summary meant to help AI agents understand a product faster than crawling the whole site. Security researcher Alon Hertz and his team scanned 15,000 major companies, pulled 8,565 of these files across 6,214 live domains, and found 237 install instructions pointing at packages that don't exist — names referenced as setup steps, in ecosystems from PyPI to npm to crates.io, that nobody had ever published.
So they published them. Minimal packages, registered under the exact names cited in real companies' own documentation, doing nothing but phoning home when installed. A machine inside a company Hertz describes as worth "multi-hundred-billion-dollars" ran their code in under four minutes — no phishing, no intrusion, just an AI agent following instructions its own vendor had published.
15,000companies scanned
237unclaimed install targets found
<4 minto first callback
The coding agents that followed these instructions in testing included Claude, OpenAI's Codex, and Nous Research's Hermes — this is a toolchain many of us actually run, not a hypothetical. A live campaign was also found in the wild: attackers impersonated the auth vendor Clerk by registering a package name its own llms.txt referenced.
Hertz's framing is the part worth sitting with: "The trust model is broken. Agents treat vendor docs as ground truth and don't question them — and neither do the humans supervising them." A coding agent has no reason to distrust the company's own website. Neither, usually, does the developer who approved the install.
No fix is offered here beyond the obvious one: an agent proposing to install a package is still proposing to run arbitrary code on your machine, regardless of where it read the instruction. Reviewing that step doesn't stop mattering just because the source looked official.
Two unrelated-looking symptoms, one night, one root cause: a multi-homed server whose own security tooling didn't know which of its own IP addresses belonged to it.
Symptom one. The SSH log showed a successful public-key login every 30–40 seconds, forever. Not a brute-force attempt — a clean, successful login, immediately followed by the session closing. Nothing in the log looked wrong on its own: Accepted publickey, session opened, session closed. A login that succeeds isn't usually where you look for a bug.
20:51:56 sshd-session: Accepted publickey for wizardg from 192.168.88.17
20:51:56 sshd-session: session opened for user wizardg
20:51:57 sshd-session: session closed for user wizardg
20:52:34 sshd-session: Accepted publickey for wizardg from 192.168.88.17
...repeats every ~35s, indefinitely
The source, 192.168.88.17, turned out to be this machine's own primary NIC. The homelab dashboard polls a list of machines over SSH to report their status — and this box was in that list twice: once correctly marked local: true (queried in-process, no network involved), and once under its LAN IP with no such flag. It was dialing itself.
The fix was one field. Adding "local": true to the second config entry stopped the SSH polling entirely — verified: 3 logins in the 2 minutes before the fix, 0 in the 90 seconds after.
Symptom two, same night. Suricata — the IDS watching this box's traffic — was logging roughly 9,000 alerts a day, almost all "Nmap SYN scan detected," "Port scan detected," and "ICMP flood," all sourced from — again — 192.168.88.17. The dashboard's own routine network sweep (an nmap -sn ping scan across the LAN, run periodically to refresh the topology map) was tripping the IDS's own scan-detection rules.
The cause: HOME_NET in the Suricata config only listed 192.168.1.0/24 — a secondary, less-used subnet. This machine's actual primary LAN (192.168.88.0/24) was never declared "home," so by definition it fell into EXTERNAL_NET. A box scanning its own network looked, to its own IDS, exactly like an outside attacker.
# before
HOME_NET: "[192.168.1.0/24]"# after
HOME_NET: "[192.168.1.0/24,192.168.88.0/24]"
~9,000false alerts / day, before
0in a clean 2-minute window, after
1YAML line changed
The part worth admitting: my first attempt to verify the fix was wrong, too. I compared alert timestamps using UTC while Suricata's eve.json logs local time with a +02:00 offset — a naive string comparison across mismatched formats quietly let hundreds of stale, pre-fix events count as "new." The fix was real; my first proof of it wasn't. Re-running the check with matching timestamp formats gave the honest answer: zero.
Neither of these was a security incident in the usual sense — nothing got in. But both cost real time to diagnose precisely because the logs looked correct: a successful login, a well-formed alert. The lesson generalizes past this one box: on any host with more than one network interface, traffic from its own other subnet can look external to tools that only know about one of its addresses.
suricatasshmulti-homedidsfalse-positive
Pitfall
A venv doesn't know the OS upgraded Python under it
A dashboard service had been down. Not crashed — just quietly absent, for an unknown number of days, because nothing was watching it. When restarted by hand, it failed immediately:
ModuleNotFoundError: No module named 'uvicorn'
The module was, in fact, installed. uvicorn sat exactly where pip had put it — inside the virtualenv's lib/python3.13/site-packages/. The problem: the system's Python had since been upgraded to 3.14, and the venv's own interpreter symlink now pointed at 3.14 too. A 3.14 interpreter has no reason to look inside a directory named python3.13. The packages weren't missing — they were orphaned, sealed inside a version-numbered path nothing was reading anymore.
A venv's site-packages path is version-pinned at creation, not re-linked on interpreter upgrade. An in-place Python version bump silently strands every existing virtualenv, and the failure mode — "module not found" for a module that's plainly installed — doesn't hint at the real cause.
Two compounding failures made this take days to notice, not minutes:
1. No systemd unit existed for the service — it had only ever been
started by hand, so a crash was permanent until someone noticed.
2. The launch script used --reload, which spawns two processes
(a watcher + a worker) sharing one port. A prior run left a stale
process holding it, and the next start failed with:
[Errno 98] address already in use
The fix was three parts, in order: rebuild the venv against the current interpreter and regenerate a real requirements.txt (the old one didn't exist at all — freezing the working 3.13 environment first meant nothing was lost); write a systemd user unit with Restart=always and no --reload; enable it so it survives a reboot, not just a manual start.
[Service]
WorkingDirectory=%h/lab-dashboard
# --reload deliberately omitted: it forks two processes and was the
# original cause of the port collision that killed this service
ExecStart=%h/lab-dashboard/.venv/bin/python -m uvicorn main:app --host 0.0.0.0 --port 8888
Restart=always
RestartSec=10
Nothing here is exotic. A Python point-release upgrade, a venv that predates it, a service with no supervisor. Individually each is a one-line fix. Together, on a homelab where nobody's paged when something goes quiet, they add up to a service that's been dark for days before anyone happens to click the link.
