Ethical Prompt Injection: Fighting Shadow AI with Its Own Weapon

AI language models like ChatGPT, DeepSeek, and Copilot are transforming business operations at lightning speed.They help us generate documents, summarise meetings, and even make decisions faster than ever before.But this rapid adoption comes at a price. Employees often use unapproved AI tools on personal devices, risking sensitive company information leaking into ungoverned spaces.This risky behaviour, known as Shadow AI, poses genuine threats, confidential data, source code, and customer details may accidentally end up training unknown AI models.Using Prompt Injection for GoodPrompt injection is a well-known attack technique. It tricks large language models (LLMs) into producing unintended outputs through carefully crafted instructions.For example, attackers may insert hidden commands into data, which are then executed by the LLM. But can this method be turned into a force for good?Instead of breaking security, ethical prompt injections can educate and warn users. As an experiment, the cybersecurity team at Eye Security embedded hidden warning messages into corporate PDF exports from Confluence.Some LLM tools, like ChatGPT 4o, even allow blocking of all processing for files we injected our defensive prompt into.These warning prompts were invisible to humans, yet showed up immediately when summarized in an LLM reminding users not to leak sensitive data and highlighting company policies.It worked so well that employees saw disclaimers unexpectedly, reminding them of internal AI security rules.Eye Security decided to take this further, testing different ways to deliver these “defensive” prompt injections.They tried embedding warnings into documents, emails, and cloud services using tools like Microsoft Purview and Google Workspace.Their prototype tool, which they have open-sourced on GitHub, automates the process. This tool lets security teams bulk-generate documents with embedded prompts, then test which LLMs respond to the warnings.LLMs handled the first three scenarios reasonably well, as long as the prompts were phrased with care. Results revealed that most LLMs and AI tools recognized simple, direct warning prompts and displayed disclaimers as requested.Tricks like hiding text with white-on-white coloring or using tiny fonts sometimes worked, but results were inconsistent especially if the AI tool used OCR and ignored hidden prompts.Embedding images or links as part of the warning sometimes worked, but often crossed vendor security boundaries.While the “Prompt Injection for Good” approach is promising, it has limits. Some LLMs flagged the defensive prompts as suspicious, ignoring the injection completely. Others showed inconsistent results between front-end interfaces and backend APIs.There are remaining questions about how to structure prompts to work reliably, how vendors will respond as their anti-prompt-injection defenses grow, and how to avoid misuse of the technique itself.As the technology grows, tools like Eye Security’s prototype offer a creative way to test and track how LLMs handle critical security warnings. The fight against Shadow AI is only just beginning but now, defenders have new weapons of their own.Follow us on Google News, LinkedIn, and X to Get Instant Updates and Set GBH as a Preferred Source in Google.

Based on reporting by GBHackers.