Knowledge
Answers, not opinions.
Narrow questions answered with measurements and sources rather than opinion. Written to be useful on their own, revised when they stop being true, and dated so you can tell.
Security and audit
- How do you security-audit a codebase full of AI slop?Not by scanning harder. AI-assisted code fails differently from human code: syntax errors and logic bugs fall, while privilege-escalation paths and architectural flaws rise sharply — defects that pass every linter because each individual line is correct. A useful audit ignores line-level patterns and goes after the four surfaces static analysis is blind to: authorisation boundaries, credentials and machine identity, dependency provenance, and what your agents are actually allowed to touch.
- Can an AI agent install a package that never existed?Yes, and the technique has a name: slopsquatting. Models invent package names at a measurable rate, the same invented names come back on rerun, and anyone can register one and wait. What changed in 2026 is counterintuitive — hallucination rates fell sharply, but the hallucinations that survived converged across models. A name that every frontier model invents is a single registration that catches every team, whichever model they happen to run.
- How do you test an LLM feature for prompt injection?Not with a checklist you run once. Prompt-injection success is stochastic: the same attack, against the same defense, on the same input, breaks through on some runs and not others. So the useful unit is not "did it block the attack" but "in how many of k tries did it hold." We built a support-bot feature and measured it — a single run made the strongest defense look near-perfect, while running the identical eight attacks eight times each broke it far more often. Test at pass@k, with adaptive attacks, and assume the number you get from one run is the best case, not the real one.
- What can an MCP server actually reach on your network?Everything you can. A local MCP server is an ordinary process launched with your user's privileges, so it can read any file you can, open any connection your machine can — localhost services, the office LAN, the cloud metadata address — and hold whatever secrets you put in its config. The tool list the client shows you limits what the model asks for, not what the process can do. We counted the official registry: 16,580 servers run locally and 38% of them ask for a secret. Treat each one as a dependency with a shell, and sandbox it — in our test, that cost nothing.
Engineering delivery
- Who actually reviews the pull requests AI agents write?Usually the person who asked for it, and usually nobody else. We measured two weeks of public GitHub in October 2026: 14.5% of pull requests were opened from a coding agent's branch, and only 1.3% of finished Claude Code pull requests were rejected, against 6.3% for everything else. People requested changes on them about eleven times less often per merge. AI bots wrote 37% of all reviews, but 93% of those were comments, not approvals. An agent's pull request opens under the prompter's own account, so the author has become the reviewer. A high merge rate measures your review process, not the model.
- The model we built on is being deprecated. What now?Migrate before the date, and treat it as a release, not a find-and-replace. We tracked every text model the OpenAI, Claude and Gemini APIs have retired: the median lived 15.7 months, and notice has roughly halved since 2023, to six months at OpenAI and 60 days at Anthropic. Some retired IDs fail loudly, some are silently rerouted to a newer model. When we re-ran one fixed test set across model upgrades, every upgrade broke between 2% and 35% of the answers that used to be right. Run your own examples on the replacement before you switch.