Ausgangspunkt für das Risiko von GenAI-Apps sind die Anwendungen selbst. In der Praxis sind Anwendungskontrollen auch der Ausgangspunkt für die Kontrolle dieses Risikos, die normalerweise als Zulassungs- oder Sperrlisten innerhalb einer Inline-SWG oder eines Proxys implementiert werden.

Die meisten Organisationen haben die Verwendung von GenAI-Apps zum Schutz ihrer Daten eingeschränkt. 77 % der Organisationen blockieren seit Juni 2024 mindestens eine GenAI-App. Dies ist ein Anstieg von 45 % gegenüber 53 % im Juni 2023.

Dieser Trend deutet auf eine gute Reife und Akzeptanz grundlegender Kontrollen rund um die Nutzung von GenAI-Anwendungen hin. Die Kontrolle darüber, welche Apps in einer Organisation verwendet werden, ist ein notwendiger Ausgangspunkt zur Risikominderung. Um wirksam zu sein, sind jedoch detailliertere Kontrollen erforderlich. Der konkrete Verwendungszweck einer Anwendung bestimmt oft, ob die Aktivität erlaubt sein sollte oder nicht. Beispielsweise sollte eine allgemeine Suche in ChatGPT erlaubt sein, die Übermittlung von Quellcode jedoch nicht.

Bei genauerer Betrachtung der blockierten Anwendungen ist auch die durchschnittliche Anzahl der für alle Benutzer in einer Organisation blockierten Anwendungen gestiegen, von 0,6 Apps im Juni 2023 auf über 2,6 Apps im Juni 2024. Dass es im Vergleich zu den Hunderten von GenAI-Apps auf dem Markt so wenige unternehmensweite Verbote für GenAI-Apps gibt, deutet auf die Beliebtheit anderer, differenzierterer Kontrollmechanismen hin, auf die wir später in diesem Bericht noch genauer eingehen werden.

Die am häufigsten blockierten genAI-Anwendungen unterliegen in gewissem Maße einer gewissen Popularität, es werden jedoch auch viele weniger beliebte Apps am häufigsten blockiert. Von den Organisationen, die genAI-Apps blockieren, blockieren 28 % Beautifal.ai (was es zur am häufigsten blockierten genAI-App macht) und 19 % blockieren Perplexity AI, die zehnthäufigste blockierte App. Sperren können oft eine vorübergehende Maßnahme sein, während neue Apps bewertet werden, um festzustellen, ob sie legitimen Geschäftszwecken dienen und für bestimmte Anwendungsfälle sicher sind.

Welche Anwendungen konkret blockiert werden, hängt von den Unternehmensrichtlinien ab. Wenn die Prozentsätze jedoch hoch genug sind, wie in der obigen Top-10-Liste, lohnt es sich für alle Unternehmen zu prüfen, ob die spezifischen Anwendungen in ihrer eigenen Umgebung verwendet werden und ob Kontrollen rund um die Anwendungskategorie angepasst werden müssen. Die folgende Tabelle zeigt, dass die am häufigsten blockierten Apps eine Vielzahl unterschiedlicher Anwendungsfälle im GenAI-Bereich abdecken.

Most Blocked GenAI App Categories

As organizations mature beyond allowed applications lists, they tend to begin enacting more fine-grained controls around the usage of the allowed applications. Unsurprisingly, DLP controls are growing in popularity as a genAI data risk control. Data loss prevention has increased in popularity from 24% in June 2023 to over 42% of organizations using DLP to control the types of data sent to genAI apps in June 2024, more than 75% year-over-year growth.

The increase in DLP controls reflects an understanding across organizations about how to effectively mitigate data risk amidst the larger, broad trend of increasing genAI app use. Within DLP policies, organizations are looking to control specific data flows, especially to block genAI prompts containing sensitive or confidential information. In organizations with data protection policies, source code accounts for nearly half (46%) of all DLP violations, with regulated data driven by industry regulations or compliance requirements at 35%, and intellectual property at 15%. Regulated data has been a top violation area pre-genAI and reinforces the challenges with manual training of users in the improper sharing of regulated data.

Posting sensitive data to genAI apps not only reflects the current priorities among organizations, but also shows the utility of various genAI apps. For example, GitHub Copilot’s popularity, coupled with Microsoft’s market share as a software development platform, may drive higher use of genAI for code generation in the future.

While obviously malicious applications or activities are served well by controls that block and alert, data risk is often more of a gray area. Coaching controls can be a highly effective technique to deal with gray areas, especially in early fast-moving technology cycles like genAI. Furthermore, organizations can use coaching to inform and refine security controls without blocking productivity with false positives and slow approval processes.

Coaching controls provide a warning dialog to the user while interacting with genAI apps, allowing them to cancel or proceed with their actions. It functions like safe browsing features built into browsers. The advantages are it provides a friendlier experience for the user, not blocking their work, but informing and enabling users to improve their behavior.

For organizations with policies to control genAI application usage, 31% of them used coaching dialogs as of June 2024 compared to 20% of organizations in June 2023, an over 50% increase in adoption.

This tracks with growing sophistication of organizations in applying similar coaching controls from other security domains to newer ones such as genAI app risk. While the growth has flattened, coaching controls are relatively new as compared to outright blocks and alerts based on DLP or application. We expect the adoption to continue growing as more organizations understand how to use coaching to manage the grayer risks associated with data.

When we break down the actual user response to the coaching dialog alerts that have occurred, we see a measure of efficacy. For all users that received coaching dialog alerts for genAI apps, in 57% of those cases, users chose to stop the action they were performing, which reduced risk by users avoiding sending sensitive data in genAI app prompts or by avoiding the use of unknown or new genAI apps. 57% is high enough to bolster the case that coaching can be an effective control to complement explicit application blocks and DLP policies. Furthermore, coaching also enables feedback. When deciding to proceed, most organizations put policies in place that require users to justify their actions during the coaching interaction.

This is not to suggest that user decision-making be the basis for security policy. Rather, it indicates that for organizations that utilize coaching, approximately half of the genAI app usage that is not being outright blocked may be further reduced by user decisions. Just as important, user decisions in response to coaching dialogs can and should inform security policy review and adjustment. While user decisions can be faulty and risky, the two categorizations form the basis for further review. Applications that a user has decided not to use based on a coaching dialog should be analyzed for an outright block list. Applications that a user decided to use should be reviewed to be put on an allowed corporate standards list or perhaps blocked if there’s a better or more acceptable app. User responses to coaching prompts could also be used to refine more nuanced policies, like DLP policies, to be more targeted in their application.

We are seeing early signs of advanced detections of suspicious user behavior with regards to data movement that were detected by behavioral detection engines. Suspicious data movement often comprises multiple indicators of suspicious or unexpected behavior by a user with respect to the user’s or organization’s normal activity baseline. The indicators might involve anomalous download or upload activity, new or suspicious sources or targets of data such as new genAI apps, as well as other suspicious behavior such as use of an unexpected or obfuscated IP address or user agent.

One example of a detected, advanced behavioral alert for suspicious data movement involves the movement of data from an organization-managed application and uploaded to an unapproved, personal genAI app.

Similar to coaching, the list of apps appearing in these alerts can also be used to prioritize control efforts with genAI application usage. For example, two commonly used apps may overlap in functionality, and that alone might prioritize efforts to reduce the number of apps.

The use of behavioral alerts show the early awareness and adoption of organizations to find harder to detect threats, which has traditionally included compromised credentials, insider threats, lateral movement, and data exfiltration activities by malicious actors.

When we look closer at the sensitive data movements, we find that the top applications where the sensitive data originated reflect popularity of corporate cloud apps with OneDrive (34%) and Google Drive (29%) at the top, followed by SharePoint (21%) and Outlook (8%) and Gmail (6%).

The top 3 apps are all cloud storage and collaboration apps and were the source of the sensitive data 84% of the time, while the major email cloud apps were the source 14% of the time. By knowing which specific applications are involved more frequently in sensitive data movements, security teams can then adjust controls appropriately, for example, placing additional DLP controls around file-sharing applications. Organizations should prioritize security risk assessment and control implementation around data movement between apps, especially moving from managed apps to unmanaged genAI apps, as it is both becoming more common and potentially having large data loss impact.