We ran an adversarial audit on ourselves: 44 findings
2026-07-15 · Vorluno
Vorluno had spent weeks shipping fast on niiko — WhatsApp onboarding, a credential vault, a CRM built in waves, a full redesign — and none of it had been checked end to end against itself. Before opening any of it to an outside audience, the answer was to turn the same adversarial process used to review a design spec into a review of the running system: assume something is broken until proven otherwise, and look across every module at once instead of trusting that each individual change's own review caught everything.
The first pass covered sixteen merged pull requests across eight adversarial dimensions — lost deferred work, event contracts, role-based access, schema and PII handling, WhatsApp end-to-end behavior, redesign leftovers, observability, documentation drift — using fifty-two finder agents plus a skeptical, default-deny verifier that re-checked every claim before it counted. The raw output was 44 findings: 41 held up under verification and three were refuted after a hand check against the actual code. The findings that stung most were confirmed, not dismissed — a missing Docker build argument that left the WhatsApp button inert in production, a disconnect button the published legal pages promised users but the UI never rendered, and a CI check that had sat red on the main branch for the entire sprint without anyone noticing — all three re-read against the real code before sealing and all three closed in the WhatsApp-and-CI pull request (#133). The three that did not survive the re-check were false alarms of one particular kind: each claimed some piece of debt or a capability was missing or lived only in throwaway code comments, and the skeptical pass found it was already recorded durably or already shipped under a different name. In total the 41 confirmed findings closed across four pull requests (#130–#133): event-sweeper hygiene, a set of role-and-ownership gaps, re-encryption after client records get merged, and the WhatsApp fixes the product's own legal pages had already promised to users.
A second round, a few days later, went further than any single module: an integration audit asking not "does this feature work" but "does anything we shipped ignore what every other module needs from it." Forty-seven agents across nine dimensions surfaced 37 findings; 21 confirmed, 16 refuted — a 43 percent refutation rate, tighter than the first pass, which reads as the process getting more disciplined rather than more trigger-happy. The one finding that could not wait for its eventual fix — PII classification with no consistent home across modules — got sealed immediately with an explicit ownership decision; the rest went into a tracked backlog attached to the designs that will eventually build them.
What repeats across both rounds is more useful than any single finding: issues cluster around ownership boundaries (who is allowed to touch what), consent and PII propagation across the seams between modules, and observability gaps where a config variable existed with nothing consuming it. None of it was one catastrophic bug — it was the accumulation of "that is someone else's problem" across a system built by one team moving fast. The categories are worth publishing; a map of exactly where a door was left open is not.
The 41-finding count is public in niiko's own changelog on purpose — sealing a coherence audit against your own system before a client or a security researcher does it for you is a decision worth stating out loud, not a result to bury under polish. Auditing yourself and publishing the number, refutations included, says more than any claim of correctness could.