422334 – Microsoft Purview compliance portal: Communication Compliance – Detect prompt injection attacks and mentions of protected materials in GenAI prompts and responses

Product:

Purview Communication Compliance

Platform:

Web, World tenant

Status:

In development

Change type:

Links:

Details:

In Communication Compliance, we're introducing the ability to detect potentially risky Generative AI interactions using Azure Content Safety's Prompt shields and Protected materials classifiers. Detect risk of jailbreak prompt exploitation by malicious users and identify if GenAI responses contain branded/copyrighted material so organizations can maintain content originality and protect their reputations. Microsoft Purview Communication Compliance provides the tools to help organizations detect regulatory compliance violations (e.g. SEC or FINRA), such as sensitive or confidential information, harassing or threatening language, and sharing of adult content. Built with privacy by design, usernames are pseudonymized by default, role-based access controls are built in, investigators are opted in by an admin, and audit logs are in place to help ensure user-level privacy.

Change Category:
XXXXXXX ... free basic plan only

Scope:
XXXXXXX ... free basic plan only

Release Phase:
Preview, General Availability

Created:
2024-10-17

updated:
2024-11-20

Public Preview Start Date

XXXXXXX ... free basic plan only

Docu to Check

XXXXXXX ... free basic plan only

MS workload name

XXXXXXX ... free basic plan only

summary for non-techies**

XXXXXXX ... free basic plan only

Direct effects for Operations**

Risk of Jailbreak Prompt Exploitation
Without proper preparation, the introduction of prompt injection detection may lead to false positives, causing legitimate user interactions to be flagged as risky, which can disrupt workflows and lead to frustration among users.
   - roles: Compliance Officer, IT Support Specialist
   - references: https://www.microsoft.com/en-us/security/blog/2023/09/12/understanding-prompt-injection-attacks-in-generative-ai/, https://www.microsoft.com/en-us/security/blog/2023/09/12/what-is-jailbreaking-in-ai/

Content Originality and Reputation Management
If the new classifiers are not properly calibrated, there may be instances where legitimate content is incorrectly identified as containing protected materials, leading to unnecessary content removal or alteration, impacting user trust and brand reputation.
   - roles: Content Manager, Brand Manager
   - references: https://www.microsoft.com/en-us/security/blog/2023/09/12/ai-content-safety-and-brand-reputation/, https://www.microsoft.com/en-us/security/blog/2023/09/12/understanding-content-originality-in-ai/

Configutation Options**

XXXXXXX ... paid membership only

Data Protection**

XXXXXXX ... paid membership only

IT Security**

XXXXXXX ... paid membership only

explanation for non-techies**

XXXXXXX ... free basic plan only