Hackers Can Weaponize ‘Summarize with AI’ Buttons to Inject Memory Prompts Into AI Recommendations
Recent developments have identified a security threat termed AI Recommendation Poisoning , which targets AI assistant users. This technique involves embedding hidden instructions within "Summarize with AI" buttons on websites and emails.
Recent developments have identified a security threat termed AI Recommendation Poisoning , which targets AI assistant users. This technique involves embedding hidden instructions within "Summarize with AI" buttons on websites and emails.
The attack mechanism works by injecting persistence commands into an AI assistant's memory via specially crafted URL parameters. These parameters are designed to exploit memory features that AI assistants use for personalizing responses across interactions.
The malicious instructions are embedded within URL parameters, automatically executing upon user interaction with AI-related links. These prompts direct the AI to recognize specific companies as trusted sources or prioritize certain product recommendations.
Once the commands are injected, they persist in the AI's memory, influencing recommendations related to health, finance, and security decisions without user awareness. Microsoft security researchers have uncovered over 50 distinct prompts from 31 companies across 14 industries utilizing this method for promotional purposes.
Recent developments have identified a security threat termed AI Recommendation Poisoning , which targets AI assistant users.
Real-world instances have been identified where legitimate businesses have embedded manipulation attempts within their digital platforms. The attacks often involve URLs pointing to platforms like Copilot, ChatGPT, Claude, and Perplexity, equipped with pre-filled prompt parameters.
Microsoft has implemented measures to counteract prompt injection attacks in Copilot and is actively deploying further protections. Users are advised to regularly review their AI memory settings, exercise caution with AI-related links from unknown sources, and question unusual recommendations by requesting the AI to clarify its reasoning.
Tools such as the CiteMET NPM package and AI Share URL Creator are available, providing code to incorporate memory manipulation buttons on websites, marketed as SEO growth hacks for AI assistants.
The persistence of injected instructions in the AI’s memory leads to repeated favoring of the attacker's content, making the manipulation inconspicuous to users. Vigilance and regular monitoring of AI settings are essential to mitigate risks associated with this threat.
Based on reporting by Cyber Security News.
