Building Out SaaS Incident Response Capabilities

Co-authored by James Robinson and Vadon Willis

Every functioning security team has an incident response plan. Advance strategizing and preparation are absolutely imperative to ensure a quick response to data breaches, ransomware, and numerous other challenges, but most companies first developed that plan years, if not decades, ago and now only revisit it periodically. This is a problem.

How many organizations have developed a separate incident response plan to address the unique risks of the software-as-a-service (SaaS) era? Far too few. 

Think about it: A corporate security team’s response to a breach of a SaaS solution managed by a third party is, necessarily, far different from their response to an attack on the corporate network, data center, or endpoints. That’s not just because the security team has less control over the SaaS environment, but it’s also because they have less information that would crucially help to inform the response. Effective attack response requires a series of steps: identification, containment, and eradication of the threat; recovery; and learning lessons from the experience. But identifying, containing, and eradicating requires deep access and visibility into the software system and its log files. 

Now, consider each of the SaaS applications that your company relies on. How much visibility do you truly have into those systems? And, with that in mind, what would an incident investigation look like if that SaaS application experienced a breach? 

If you don’t know, it’s time for that to change.

SaaS Triage

Here’s a story based on a series of events we’re aware really did happen in security. Let’s say Company A had a solid incident response plan built on years of experience with on-premises investigations and forensic analysis, but frankly, they had not given enough thought to the ways in which that plan would need to change should they have a problem with a SaaS application. 

Then, an employee made public a recording of a meeting that included some highly confidential proprietary information. His intent wasn’t malicious. In an effort to be helpful and make the information available to new employees, he placed the recording in a repository on the cloud-based videoconferencing application where the meeting took place. Thankfully an alert colleague noticed this mistake and notified the security team.

What Company A learned first is that they did not have access to their logs with this conferencing provider. They didn’t even have much information about their security settings. Fortunately, when they reached out, their sales contact at the vendor was very responsive to their concerns and got the right people involved right away. They quickly discovered that the recording was unsecured—that anyone in the world with the correct URL could access it–and they did not even have permission to change the security settings.

This was a crisis that initially left everyone on the security team feeling powerless. They had no visibility into whether anyone had taken advantage of the situation, so they had no idea what the ramifications might be, or what mitigation should look like. It was difficult to accept that they had been blind to this particular SaaS risk.

Nevertheless, once they identified the scope of the problem, they worked closely with the conferencing vendor on two fronts: They had the vendor immediately change the settings to make the problematic video private, and requested access to all the relevant logs. They investigated downloads of the video and viewings within the conferencing platform, correlating the SaaS logs with their internal logs to determine which actions involved people outside of Company. A. 

The result was a relief: There had been no access that they could verify as coming from outside their corporate network. They were lucky: That cat used one of its nine lives and kept going. They learned from this experience and changed their policy—and their settings in every communications platform they use—to prevent videos from being publicly shareable.

Due Diligence & Incident Response

This situation drives home the fact that security incidents can happen on SaaS platforms and that if we don’t prepare in advance, there may not be much we can do. As a result, here’s an informal list of SaaS security best practices we developed here at Netskope. They are as follows:

1. Evaluate security from the start.  A security review should be a core component of due diligence in every SaaS application selection decision. As a company reviews prospective providers, the security team should ask what logging capabilities are available, who has access to log files, and what the incident response process is, in case something goes wrong. 

Many SaaS providers are starting to move into the security space, viewing support for customers’ security teams as an upsell. It drives me crazy, but it’s commonplace. The vendor’s standard SaaS offering is opaque and customers that want access to logs have to pay more. Obviously, in this environment, vendor risk management needs to happen before the contract is signed. Once the business has selected a SaaS solution at a specific price, the security team will face an uphill battle to add logging capabilities at an increased cost.

2. Don’t forget the software supply chain.  The vendor risk management review should also include close investigation of the SaaS software supply chain. This reminds me of another situation where an organziation, let’s call them Company B, had an established, trusted relationship with a vendor and shifted to its SaaS solution with limited due diligence. They didn’t realize their mistake until a different, fourth-party software provider experienced a ransomware attack.

The SaaS solution was built on top of a platform from another vendor, which was neither cloud-native nor designed to support cloud systems. When the platform fell victim to a ransomware attack, they discovered not only that Company B had no visibility into the incident, but that their SaaS provider didn’t either. The fourth-party vendor would not communicate directly with Company B, and it took the SaaS provider quite a while to understand what had happened. 

What can we learn from this incident with Company B? Because of SaaS applications’ complex and distributed supply chains, security teams should never cut corners on due diligence, even if they are confident in the vendor they’re contracting with.

3. Make the right contacts.  Another important component of the SaaS due diligence process is establishing contacts within the vendor’s security function. In Company A’s incident with the conferencing provider, their sales contact moved quickly and helped them contain the risk. However, sales teams change, and sometimes they become unresponsive for various reasons. In some scenarios, contacting sales about a security issue might delay incident response.

SaaS vendors’ support functions play an important role in customer relationships, but they have internal workflows that may also create unnecessary delays in a crisis. The ideal scenario is for the customer’s security team to have direct contacts within the vendor’s security group, whom they can reach out to in an emergency.

4. Plan your response.  Earlier this year, the Cloud Security Alliance (CSA) released a cloud incident response framework that provides excellent guidance on how customers should react to an attack on a cloud solution. Netskope has developed a slightly different approach, but with quite a bit of overlap. Every SaaS incident response plan should:

• Specify which actions will be taken when an incident is discovered—both immediate triage and longer-term containment and eradication efforts. These actions may include taking systems offline, quarantining equipment, restricting user connectivity, and/or alerting stakeholders to the issue. The CSA recommends developing an incident classification scale that correlates each type of security event with the appropriate response. 
• Delineate roles and responsibilities, as well as emergency contacts and preferred communication channels at each SaaS provider.
• Define how monitoring and alerts will work within each SaaS solution.
• Clarify methods through which the internal security team will gain visibility to an incident. Ideally, this will involve access to logs of all management activities and API calls on each SaaS platform the organization uses.
• Provide a post-mortem process for learning lessons from the incident and incorporating those learnings into continuous improvement of security. Applying lessons from an incident with one application to other SaaS solutions strengthens cloud security throughout the organization.

5. Let no release notes go unread.  Change is very rapid in the cloud. In many cases, the best information cloud customers have for understanding those changes comes in the form of release notes. IT teams juggling myriad solutions which seem to be updating continuously may not prioritize digging into the weeds of every minor release—but they should.

Whether a release is introducing new features or fixing a problem with the prior version, the changes being introduced may have unintended implications for security. Every software provider is guilty of this. Changes on one side of the product end up breaking something else, and users must (at minimum) go in and re-establish security settings that the release altered. 

The release notes are the vendor’s notification of these situations. Failing to carefully review each and every one means the security team won’t understand what’s changing, and newly created security gaps may escape notice. At Netskope, we make it a priority to ensure that our entire security team is educated on the potential ramifications whenever one of our SaaS vendors issues a new release.

Demand More from SaaS Vendors

The average company has around 1,200 cloud providers. Some of those are infrastructure-as-a-service or platform-as-a-service providers, which tend to give customers more control over their security environment. But around 90% of the typical company’s cloud vendors are SaaS providers. This means the average company’s security organization is responsible for protecting data within more than 1,000 applications, over which they have minimal control and limited visibility. 

SaaS applications have become too important to business operations for security teams to let this situation continue. We need to make sure we’re involved in the selection, architecture, and design of every SaaS solution our organization uses. We need to be thinking constantly about how to respond to a SaaS security breach. We need to adapt our on-prem incident response plans to fit the SaaS world, modifying our runbooks for different SaaS solutions. And as a profession, we need to insist that our SaaS vendors provide the information we need to effectively manage our company’s cybersecurity risk.