O ponto de partida para o risco do aplicativo genAI são os próprios aplicativos. Na prática, os controles de aplicativos também são o ponto de partida para controlar esse risco, normalmente implementados como listas de permissão ou bloqueio em um SWG ou proxy em linha.

A maioria das organizações restringiu o uso de aplicativos genAI para proteger seus dados, com 77% das organizações bloqueando pelo menos um aplicativo genAI em junho de 2024, o que representa um aumento de 45% em relação aos 53% das organizações em junho de 2023.

Essa tendência indica boa maturidade e adoção de controles básicos em relação ao uso de aplicativos genAI. Controlar quais aplicativos são usados em uma organização é um ponto de partida necessário para reduzir os riscos. No entanto, serão necessários controles mais granulares para serem eficazes. O uso específico de um aplicativo geralmente determina se a atividade deve ser permitida ou não. Por exemplo, uma pesquisa geral no ChatGPT deve ser permitida, enquanto o envio do código-fonte não deve.

Analisando mais detalhadamente os aplicativos bloqueados, o número médio de aplicativos bloqueados para todos os usuários de uma organização também tem aumentado, de 0,6 aplicativo em junho de 2023 para mais de 2,6 aplicativos em junho de 2024. O fato de haver tão poucas proibições de aplicativos genAI em toda a organização, em comparação com as centenas de aplicativos genAI no mercado, aponta para a popularidade de outros controles mais matizados, que discutiremos em mais detalhes posteriormente neste relatório.

Os aplicativos genAI mais bloqueados estão de certa forma relacionados à popularidade, mas um número razoável de aplicativos menos populares são os mais bloqueados. Das organizações que bloqueiam aplicativos de genAI, 28% bloqueiam o Beautifal.ai (tornando-o o aplicativo de genAI mais comumente bloqueado) e 19% bloqueiam o Perplexity AI, que é o décimo aplicativo mais comumente bloqueado. Muitas vezes, os bloqueios podem ser uma medida temporária, pois os novos aplicativos são avaliados para determinar se atendem a alguma finalidade comercial legítima e se são seguros para determinados casos de uso.

Os aplicativos específicos que são bloqueados variam de acordo com a política da organização, mas quando as porcentagens são altas o suficiente, como na lista dos 10 principais acima, vale a pena que todas as organizações analisem se os aplicativos específicos são usados em seu próprio ambiente, bem como se devem ajustar os controles em torno da categoria de aplicativos. A tabela a seguir mostra que os aplicativos mais bloqueados abrangem uma variedade de casos de uso diferentes no espaço da genAI.

Most Blocked GenAI App Categories

As organizations mature beyond allowed applications lists, they tend to begin enacting more fine-grained controls around the usage of the allowed applications. Unsurprisingly, DLP controls are growing in popularity as a genAI data risk control. Data loss prevention has increased in popularity from 24% in June 2023 to over 42% of organizations using DLP to control the types of data sent to genAI apps in June 2024, more than 75% year-over-year growth.

The increase in DLP controls reflects an understanding across organizations about how to effectively mitigate data risk amidst the larger, broad trend of increasing genAI app use. Within DLP policies, organizations are looking to control specific data flows, especially to block genAI prompts containing sensitive or confidential information. In organizations with data protection policies, source code accounts for nearly half (46%) of all DLP violations, with regulated data driven by industry regulations or compliance requirements at 35%, and intellectual property at 15%. Regulated data has been a top violation area pre-genAI and reinforces the challenges with manual training of users in the improper sharing of regulated data.

Posting sensitive data to genAI apps not only reflects the current priorities among organizations, but also shows the utility of various genAI apps. For example, GitHub Copilot’s popularity, coupled with Microsoft’s market share as a software development platform, may drive higher use of genAI for code generation in the future.

While obviously malicious applications or activities are served well by controls that block and alert, data risk is often more of a gray area. Coaching controls can be a highly effective technique to deal with gray areas, especially in early fast-moving technology cycles like genAI. Furthermore, organizations can use coaching to inform and refine security controls without blocking productivity with false positives and slow approval processes.

Coaching controls provide a warning dialog to the user while interacting with genAI apps, allowing them to cancel or proceed with their actions. It functions like safe browsing features built into browsers. The advantages are it provides a friendlier experience for the user, not blocking their work, but informing and enabling users to improve their behavior.

For organizations with policies to control genAI application usage, 31% of them used coaching dialogs as of June 2024 compared to 20% of organizations in June 2023, an over 50% increase in adoption.

This tracks with growing sophistication of organizations in applying similar coaching controls from other security domains to newer ones such as genAI app risk. While the growth has flattened, coaching controls are relatively new as compared to outright blocks and alerts based on DLP or application. We expect the adoption to continue growing as more organizations understand how to use coaching to manage the grayer risks associated with data.

When we break down the actual user response to the coaching dialog alerts that have occurred, we see a measure of efficacy. For all users that received coaching dialog alerts for genAI apps, in 57% of those cases, users chose to stop the action they were performing, which reduced risk by users avoiding sending sensitive data in genAI app prompts or by avoiding the use of unknown or new genAI apps. 57% is high enough to bolster the case that coaching can be an effective control to complement explicit application blocks and DLP policies. Furthermore, coaching also enables feedback. When deciding to proceed, most organizations put policies in place that require users to justify their actions during the coaching interaction.

This is not to suggest that user decision-making be the basis for security policy. Rather, it indicates that for organizations that utilize coaching, approximately half of the genAI app usage that is not being outright blocked may be further reduced by user decisions. Just as important, user decisions in response to coaching dialogs can and should inform security policy review and adjustment. While user decisions can be faulty and risky, the two categorizations form the basis for further review. Applications that a user has decided not to use based on a coaching dialog should be analyzed for an outright block list. Applications that a user decided to use should be reviewed to be put on an allowed corporate standards list or perhaps blocked if there’s a better or more acceptable app. User responses to coaching prompts could also be used to refine more nuanced policies, like DLP policies, to be more targeted in their application.

We are seeing early signs of advanced detections of suspicious user behavior with regards to data movement that were detected by behavioral detection engines. Suspicious data movement often comprises multiple indicators of suspicious or unexpected behavior by a user with respect to the user’s or organization’s normal activity baseline. The indicators might involve anomalous download or upload activity, new or suspicious sources or targets of data such as new genAI apps, as well as other suspicious behavior such as use of an unexpected or obfuscated IP address or user agent.

One example of a detected, advanced behavioral alert for suspicious data movement involves the movement of data from an organization-managed application and uploaded to an unapproved, personal genAI app.

Similar to coaching, the list of apps appearing in these alerts can also be used to prioritize control efforts with genAI application usage. For example, two commonly used apps may overlap in functionality, and that alone might prioritize efforts to reduce the number of apps.

The use of behavioral alerts show the early awareness and adoption of organizations to find harder to detect threats, which has traditionally included compromised credentials, insider threats, lateral movement, and data exfiltration activities by malicious actors.

When we look closer at the sensitive data movements, we find that the top applications where the sensitive data originated reflect popularity of corporate cloud apps with OneDrive (34%) and Google Drive (29%) at the top, followed by SharePoint (21%) and Outlook (8%) and Gmail (6%).

The top 3 apps are all cloud storage and collaboration apps and were the source of the sensitive data 84% of the time, while the major email cloud apps were the source 14% of the time. By knowing which specific applications are involved more frequently in sensitive data movements, security teams can then adjust controls appropriately, for example, placing additional DLP controls around file-sharing applications. Organizations should prioritize security risk assessment and control implementation around data movement between apps, especially moving from managed apps to unmanaged genAI apps, as it is both becoming more common and potentially having large data loss impact.