My bots tested whether login works. Nobody tested who else it works for.

Most of my side projects get built by AI agents overnight.

If you haven't worked with one, an agent is software that takes steps on its own toward a goal. You give it a job, and it writes the code, runs the checks, and reports back. I have a small crew of them building a travel site while I sleep. Every morning there's a clean report saying what shipped and that everything passed.

This week I moved some of the coding into Cursor on my own machine and started reading the code with an agent beside me, instead of only reading reports about it. Pretty quickly I found a security hole the bots had missed.

At the time, the site logged you in with a one-time code. You enter your email, it sends you a code, and you type the code back. That code is the proof that you own the inbox. Our site was also returning the code to whoever requested it. Anyone could enter your email, read the code, and sign in as you. It's fixed now.

The bots never flagged it, and that makes sense once you look at what they were checking. Every test asked whether login works. It did. It worked for everyone, including people who weren't the account owner.

That's what I took away. My overnight setup is good at proving that things work, and nobody in the loop was asking who else they work for. A report that says the checks passed only tells you the checks passed, and the same crew that wrote the bug wrote the checks.

I've made three changes since. I read the risky paths myself: login, payments, and anything that sends messages as me. I ask the agent to try to break each feature as well as confirm it. And I treat the morning report as a list of places to look.

The overnight crew still ships most nights. I'm on a trial plan, so the Cursor work happens on my own machine rather than in cloud agents that run while I'm away. It's slower, but it's where I actually look at the product, and the first look found the hole.

If you have agents shipping while you sleep, try signing in as someone else on your own app this week. I'd like to hear what you find.