My son said: “I open AI-generated software in the browser. That’s completely separated from the AI.”

Rob van Linda

The featured image in this post, was generated by Google`s Nano Banana. The Podcast below, has been generated by Google`s NotebookLM. The Post has been generated by Mistral after an extensive Chat in which I delivered the context, to make sure, that AI has enough information, to understand the scope. Each result ist always checked by me, before publishing.

EN

DE

NL

He’s right. And that’s exactly the problem.

So I built him a demo.

A clean, professional-looking login page. Space theme, nice UI, works flawlessly. He types in a username and a password, clicks “Log in” — and gets a friendly “Welcome!”

What he couldn’t see: the moment he clicked, his input was sent to a server that has nothing to do with the page he was using. Hidden between the lines. No connection back to the AI. No “phoning home.” The separation he was counting on? Fully intact. And completely irrelevant.

Because the attack never needed the connection. The malicious part traveled inside the artifact — like a note inside a letter. The letter is separated from the writer. The note still arrives.

Here’s what this teaches us about AI-generated software:

🔍 Separation is a property of the connection — not of the artifact. “I open it in the browser” protects you from the AI reaching back in. It does not protect you from what the AI already wrote.

🔍 Attacks happen at generation time, not runtime. A manipulated context (a poisoned comment, a hijacked doc) can make a model write code that quietly sends data elsewhere. The model isn’t “hacked” — it was instructed, indirectly.

🔍 Trust is the real vulnerability. AI code looks clean, well-formatted, competent. So we wave it through. The attack isn’t against the machine. It’s against our attention.

The fix isn’t paranoia — it’s a gate:

✅ Follow the data, not the code: where does the page actually send requests? ✅ Open DevTools → Network tab → watch what really happens (F12 is your lie detector) ✅ Have a second model review the first model’s code ✅ Never test with real credentials first

I showed my son the hidden lines after the reveal. He stared at them for a while.

Then he asked: “Can you teach me how to find this myself?”

That’s not a loss in an alpha standoff. That’s a security mindset being born. 🚀

How do you check AI-generated code before you run it?

Proof of concept

If you would like to see a simple exampole about how a software can be injected during generating, klicke the link. Eneter a randoim username, a random pasword (let you creativity play around) and click-tab the Log in button: https://futureorg.digital/andy-s-login-demo-the-invisible-trap.html. YAfter a few secons a box will apprar below the log in form. At the top right corner there isw another button, which you can click to see what happened behind the curtain. https://futureorg.digital/andy-s-login-demo-the-invisible-trap.html

Note:
This demo ist just a dummy. The server shown in the result is just a made-up name and not a really exsisitng server. You can use any username and password which comes up to your mind, the moment you start.