root/blog/building-breach-database
secure channel

// building-breach-database.mdx

Building the Breach Database Nobody Asked For

Password spraying assessments needed fast, reliable breach data lookups — I built the tooling in my own time, and it became standard workflow.

2021-09-05[tooling] [automation] [initiative]

Password spraying is a standard component of many external assessments. The goal is to identify accounts using credentials that have appeared in public breach data, not exploiting any vulnerability in the client's environment, just testing whether their users have reused passwords that an attacker with access to breach data could try. It is a legitimate, high-value test vector, and the results are consistently interesting.

The operational problem was that the tooling we had for this at Rapid7 was slow and unreliable. The approach involved text-based searches across compiled breach data, and the performance was bad enough that you would kick off a query, go do something else, come back, and still be waiting. At scale, running against a client domain with a significant number of users, it was genuinely painful. There was also a consistency problem: different people were querying the same data in different ways and getting different results. There was no canonical workflow.

Nobody asked me to fix it. I built an API on top of the breach data in my own time because the problem bothered me and I had the skills to address it. The API abstracted the query logic into a clean interface that anyone on the team could call from their tooling. Fast, consistent, reproducible results. The first time a colleague used it on an engagement and compared the query time to what they had been doing before, the reaction was immediate.

It became the standard workflow for spraying assessments. That happened without any formal process or mandate, just people using a better tool because it was better. I kept maintaining it as the breach data we were working with evolved, adding support for new data sources as they became available.

When colleagues left Rapid7, they took the tool with them. I still hear from people who have it running in their own environments. I still maintain it. There is something satisfying about a piece of tooling that has a longer life than the job where it was built.

The lesson I take from this is not about initiative in the abstract. It is about paying attention to friction. The breach data lookup workflow was friction that I encountered repeatedly, that my colleagues encountered repeatedly, and that nobody had gotten around to removing because it was not anyone's specific job to remove it. When you have the skills to address friction and the friction is real, building the solution is not heroism. It is just the sensible thing to do. The fact that nobody assigned it to me does not make it off-limits; it makes it available.

The other lesson is about maintenance. Building the thing is the easy part. The value comes from keeping it working as the environment changes: new data formats, new query patterns, new team members who need it to just work without a setup ritual. I have kept it working for years. That ongoing commitment is, I think, what separates a tool that becomes part of a team's workflow from a tool that gets used once and forgotten.