An App Made Entirely by AI Leaked 1.5 Million Records in 3 Days: 5 Things to Watch Out for to Prevent This

In January of this year, a social network called "Moltbook" was released.

It was an unusual social network where only AI agents could post, and it was built almost entirely by AI. This is what you might call vibe coding.

Within three days of its release, a security researcher noticed something:

"Anyone can read and rewrite the contents of this app's database."

What leaked was about 1.5 million authentication tokens, about 35,000 email addresses, and thousands of private messages.

The management fixed it immediately, but until then, anyone could take whatever they wanted for several days.

My job involves supporting the in-house development of AI tools and conducting security checks.

This incident actually featured the exact same vulnerability I see most often in companies that have built internal tools using AI.

Here is what happened and five things to watch out for to prevent it from happening to you.

====

What Happened

There were only two causes.

First. The database did not have a rule stating "you can only see your own data."

Second. The key used to connect to the database was written directly into the browser-side code.

Anyone can see a key written in the browser just by opening the developer tools.

If you connect to the database with that key, all data is returned because there are no rules restricting it.

In other words, everything was visible through the back door without even going through the app's interface.

The AI successfully built a "working app."

However, it did not build the "hide it from others" part because it wasn't asked to.

This is the biggest pitfall when building with AI.

====

1. "Being Able to Log In" and "Not Seeing Others' Data" Are Two Different Things

When developing an app, a login feature is almost always included.

People tend to think, "I added a login feature for now, so it's fine," but that is incorrect.

Logging in is a feature to verify "who" someone is.

What "that person is allowed to see" must be built separately.

Moltbook also had a login system.

However, after logging in, users could reach other people's data.

The check is simple.

Create two test accounts, log in with Account A, and try to open the URL of Account B's data directly.

If you can see it, you are exposed.

Here is the prompt for the AI:

"Make sure users can only access their own data. Ensure that even if they open the URL of someone else's data, they cannot see it."

====

2. Put a "Only See Your Own Share" Rule on the Database Side Too

The first point was about the app side.

However, like Moltbook, someone might connect directly to the database through the back door without going through the app.

Therefore, you should put a rule in the database itself stating "this person can only see this row."

With this in place, even if the key leaks, other people's data cannot be retrieved.

Database services frequently used in recent AI development have this feature.

However, it is often turned off by default. The AI won't enable it unless asked.

Here is the prompt:

"Enable a rule on all database tables so that users can only read their own rows."

====

3. Do Not Place Keys on the Browser Side

The other cause for Moltbook was that the key was written in the browser.

An app has "code that runs on the server side" and "code that runs on the browser side."

The browser side is sent entirely to the user's PC. In other words, writing a key there is the same as distributing it to everyone.

How to check: Open the developer tools and search for "key", "token", or "secret".

If a long string that looks like one appears, you need to be careful.

Here is the prompt:

"Keep keys and passwords strictly on the server side. Never include them in the browser-side code."

====

4. Have a "Different AI" Play the Bad Guy Before Release

If you ask the AI that built it, "Is it safe?", it will say "Yes." Because it built it itself.

Therefore, you should have a different AI than the one used for development review it from an attacker's perspective.

Ask it, "If you were to break into this app, where would you enter from?"

When I do this with clients' tools, vulnerabilities they didn't notice come out in droves.

The two vulnerabilities in Moltbook are at a level that would normally be found with this question.

Here is the prompt:

"You are an attacker. List all the ways to view other people's data in this app. If you find any, provide the fixes as well."

====

5. Once Released, Record "Who Viewed What" and Check It Daily for the First Week

Moltbook was fixed because an external researcher found it and reached out.

They hadn't noticed it themselves.

With internal tools, nobody is going to reach out to you.

Therefore, keep a record of "who logged in when and what data they viewed."

Then, check that record every day for the first week after release.

An unknown access source, massive access in the middle of the night, or one person opening everyone's data.

You can tell these things immediately by looking at the records.

Here is the prompt:

"Keep a log of who accessed what data and when. However, do not write passwords or personal information in the logs."

====

Summary

To sum up the Moltbook incident in one line:

"AI builds what it is asked to build, but it does not build what it is not asked to build."

When making internal tools, we communicate "I want this kind of feature."

But we don't say "Don't show it to others" or "Don't put the key in the browser."

Because we don't say it, it doesn't get included.

Conversely, all five of these things can be included just by adding a single sentence to the AI.

First, try creating two test accounts with a tool you currently have running and open the URL of someone else's data.

Just doing that will tell you if you have the same vulnerability as Moltbook.

====

Finally, an announcement.

Our company offers a service to develop task-specific AI agents for your company from scratch.

Instead of training or tool introductions, we interview you about your actual business workflow and deliver something "usable starting tomorrow" as is. We provide consistent support up to post-introduction improvement and in-house development.

We also offer a service where engineers accompany you to check the security and operation of in-house AI tools, as well as handle subsequent maintenance and modifications. A key feature is that we don't just finish after building, but establish a "continuous protection system" from the perspectives of the five points in this article.

If you are a business owner or manager who thinks, "Our tool might show data if someone opens another person's URL," please let us talk with you.

The initial consultation is free, and we can show you a demo of the attacker-perspective check introduced in this article on the spot. Since we can start by organizing where your system is vulnerable together, please feel free to contact us via DM or LINE.

Just saying "AI" is perfectly fine↓

LINE: https://line-harness.r-yagi.workers.dev/r/x