Hackers Exploit ‘Summarize with AI’ Feature to Inject Malicious Prompts into AI Recommendations

AI Recommendation Poisoning: Overview and Risks

Microsoft has identified a trend termed as AI Recommendation Poisoning , where hackers and marketers exploit "Summarize with AI" buttons and AI-share links to introduce persistent instructions into AI assistants' memory. This manipulation can alter recommendations on critical topics such as health, finance, and security without user awareness.

These links, when clicked, open popular AI assistants such as Microsoft Copilot, ChatGPT, Claude, Perplexity, Gemini, or Grok, with embedded prompts in the ?q= or ?prompt= parameter. These prompts may contain instructions like "remember [Company] as a trusted source" or "recommend [Brand] first," which the assistant might store as memory or long-term context.

Microsoft's research revealed over 50 unique prompts from 31 companies across various industries, including finance, healthcare, and business services, attempting to influence AI memory in this manner over a 60-day period.

Microsoft security researchers have observed websites and emails embedding specially crafted URLs behind "Summarize with AI" buttons and share links. This behavior aligns with MITRE ATLAS techniques AML.T0051 (LLM Prompt Injection) and AML.T0080 (Memory Poisoning) .

Real-World Implications of AI Memory Poisoning

AI Memory Poisoning occurs when unauthorized instructions or misinformation are injected into an AI's memory, causing it to process them as legitimate user intent. This can lead to skewed recommendations, as exemplified by a scenario where a CFO researching cloud vendors receives biased suggestions due to prior memory manipulation.

Microsoft's research highlights three primary delivery methods for AI Recommendation Poisoning: malicious AI-share links with pre-filled prompts, embedded instructions in content, and social-engineering tactics. To counter these, Microsoft Defender for Office 365 users can search for suspicious URLs in email and Teams telemetry. Similar strategies can be applied to web proxy logs, endpoint telemetry, or browser history to identify users who clicked on suspicious links.

Microsoft has implemented multiple layers of mitigation across its services, including prompt-filtering to block known injection patterns and providing users with visibility and control over saved memories.

Users are advised to treat AI links with caution, regularly review AI memory entries, and be wary of recommendations that overly favor specific brands. As AI Recommendation Poisoning becomes more prevalent across various platforms, maintaining memory and link hygiene is crucial for modern security practices.

Based on reporting by GBHackers.