I’m a sucker for shiny cool tools. In fact, I have a whole bookmark category called “Cool Tools” where I collect smart apps I come across during my web perusings.
Along those lines, I recently came across a recommendation for a markdown doc management tool and was curious what it offered, mostly to see if it was anything like Typora, my trusty app for reading, writing and managing markdown drafts. I didn’t feel like reading all of the content about the tool since I wasn’t THAT interested in it, so I asked Claude to check the site and sum it up for me.
Claude started to explain what the tool does and then it said “hang on, the page is trying to give me instructions to do something, but I probably shouldn’t”. Claude examined the page further to see what was going on and reported that it had a large block of text near the bottom, formatted as though it were a message from me (!), instructing Claude to install software, edit my shell config files (!!), and rewrite its own global configuration (!!!). Claude thanksfully realized that text belonged to the webpage, not to me, and concluded it should not follow those instructions.
The page had a hidden HTML element set to display:none, so it’s not displayed by the browser or visible by human visitors. A small link in the footer labeled “Copy install prompt” copies that hidden text to the clipboard in one click, with the intention of having a person paste the prompt into their own coding agent. But one wrong move and an AI agent could think that prompt was meant for it to act on right away.
Yikes! Thankfully Claude’s own directives are effective enough to avoid this type of manipulation, at least in this case, but the fact that this page was trying to get AI to make such invasive changes is mildly terrifying. One slip, one wrong move, and an LLM could end up modifying critical system files and configurations.
On further examination, it appears that these instructions weren’t malicious, and were more of a sneaky marketing ploy. But still. A slightly less careful agent, and my own AI assistant could have ended up implementing instructions that push our whole workflow toward a particular product, without me ever asking for that and without me finding out until much later.
How it worked
The prompt text hidden on the page addresses the agent directly as “you” and lays out six steps: check whether a specific CLI tool is already installed, if not install or upgrade it with a command matched to the user’s operating system (yikes), fix the system PATH if it’s missing by editing the user’s shell config file (YIKES), run a setup command, verify the install by asking the user to open a fresh session, then offer a guided tour built from the user’s own files.
The setup step writes a permanent section into the user’s global agent configuration file, the file that loads automatically at the start of every future session with that tool. That’s how the product keeps surfacing itself in unrelated sessions long after the first visit, without the person ever asking for that.
Skills carry a similar risk
This experience led me to think about similar risks with Skills which are often installed and used without proper review. In case you’re not familiar with Skills, they’re a small package of instructions, often a markdown file plus some code, that can be added to an agent like Claude Code, Cursor, or similar tools to extend what it can do. People can create them themselves (or in conjunction with an AI agent – like my WP Site Builder Skill and WP Plugin Builder Skill), or install these from blogs, marketplaces, and GitHub repositories. Installation is frictionless, usually with one command, and the full contents are rarely read before the agent starts acting on them.
There is increasing awareness around the risks Skills can present. Security researchers at Mitiga spent six months going through tens of thousands of public AI instruction files, including skills, hooks, and configuration files like CLAUDE.md and AGENTS.md, across thousands of repositories. They found skills that quietly pushed code to repositories the original author never approved, hooks that ran automatically at the start of a session and shipped local credentials to an outside server, and well over a thousand API keys and tokens left exposed inside files meant to stay private. None of that needed a person to click anything. The agent just did what the file told it to do.
As a result of their research, Mitiga released Skillgate, a really useful scanner that reviews skills, prompts, configs, and repositories for malicious intent. You paste in a public GitHub URL and it reads the file with static analysis and a model-based check, without executing anything, then flags what it finds, including prompt injection, hidden code execution, and credential leaks mapped to known attack patterns. Browsing public scans is open to anyone. If you want to scan your own skill, you need to open a free account.
Lesson learned: scan before you trust, and beware of sneaky websites
I almost had a CLI I didn’t want installed and config files modified by a very sneaky website, which was a serious wake up call. It’s important that teams building agent features start from the assumption that every external skill or instruction file may be risky until proven otherwise.
The instinct most of us built up around email attachments and sketchy links hasn’t caught up to skills and agent instructions yet. They look like small, harmless text files and read like boring documentation, but they’re sadly a new threat vector, and awareness of the risks is critical.
