ayberk.ninja ayberk.ninja

Incident Response On AWS

17 minutes to read


As in on-prem environments, security in cloud environments should be considered as a whole. As it can happen in any environment, hacking cases can occur in AWS environments. We see examples of this situation from time to time. In this article, I will not go into the details of Incident Response processes. I will mostly explain how you can run Incident Response processes on AWS.

Is It So Different from Incident Response?

In essence, no. Of course there are some differences. But it is important to remember that cloud systems are not much different from normal computer systems. Without going deeper, I should mention that I will not talk about Incident Response processes in this article. I will talk about how Incident Response processes are implemented in AWS. If you do not have basic knowledge about Incident Response, I recommend you take a break here and read a short articles about Incident Response processes.

AWS Incident Manager

There is a service offered by AWS that allows you to easily manage Incident Response processes. Incident Manager. With the Incident Manager service, you can plan your Incident Response processes, define Runbooks, send notifications to relevant teams and review incident details for up-to-date information during an incident. Incident Manager does all this by leveraging other AWS services.

AWS Incident Manager

Let’s examine how Incident Manager is configured and how to use it.

Setting Up Replication

After entering Incident Manager’s panel, we can start configuring it by clicking the Set up button under General Settings. Then let’s continue by confirming the Terms and conditions.

AWS Incident Manager Replication

At this point, we make general adjustments to Incident Manager. In the Regions field, we determine in which Regions we will use Incident Manager. We need to select at least one Region. But there is no upper limit to the number of Regions we will select.

You can use the KMS Encryption field to guarantee that the data stored in Incident Manager is protected and cannot be changed without deletion. Once you have done this, it can take up to 5 minutes to set up Replication (but it usually takes a few seconds). You can get yourself a cup of coffee.

Setting Up Contact Details

This stage is optional. You do not have to set it. But I recommend you to follow every step in Incident Manager to run a good Incident Response process. This is the area where we set the people and communication channels to respond to the Incident.

In the Contact details field, the name of the person who will deal with IR processes and an alias are specified.

In the Contact channel field, we specify how to contact the person we specified in the Contact details field in case of a case. In this field, we can specify a communication channel via E-mail, SMS, or Voice. We can also specify more than one communication channel. My recommendation is to specify at least two communication channels. Thus, if the relevant person cannot be reached through one channel, they can be reached through the other channel.

AWS Incident Manager Contact Detail

In the Engagement area, you can specify when to contact the relevant contact(s) in case of an incident. Note that after you have set up the contact channels, a validation will be sent for each contact channel you have set up.

AWS Incident Manager Contact Verification

If you want to set more than one contact you have to repeat the same procedure.

Setting Up Escalation Plans

Creating an escalation plan is optional, just like creating a contact. But this time, if you want to create an escalation plan, you need to have already created a contact. You can use it in situations where you want more than one person to deal with a case at the same time. You can designate more than one person and forward the case to the other person(s) in case the first person does not respond to the case.

In the Escalation plan details field you must assign a name and alias to the related plan. In the Stages field, you can specify which people will deal with the case, and with the Duration parameter, you can specify how long in minutes it will be escalated to the next Responder. Duration must be 30 or less than 30.

If you want to create more than one Escalation plan you should repeat the same steps.

AWS Incident Manager Escalation Stages

Setting Up Response Plan

I think the most crucial part is the Response Plan. You can use this area to plan how to respond to incidents, determine the severity of incidents, determine which contacts to contact, select metrics to track, and determine the automated runbooks to start.

As always, we start by specifying a name and alias for the Response plan in the Response plan details field. The values in the Incident Defaults field and their descriptions are as follows:

AWS Incident Manager Incident Defaults

The chat channel field is optional but very useful. Select a chat channel for responders to interact during the case. Currently, only Slack and Chime are supported. In order to use this area, you must first configure a Chatbot Client. For more detailed information, you can review the AWS document.

I explained the Engagements section in the previous chapter, so I will continue by skipping this section.

Runbooks allow you to automate some processes. You can create and use a Runbook, use one of the Runbooks created by AWS, or use another Runbook that has been shared with you. I should mention that for Runbooks to work, you must assign an IAM role with ssm:StartAutomationExecution authorization. Also, if you are going to use Runbook in Cross-Accounts, you must have sts:AssumeRole role.

Creating A New Runbook

Creating a new Runbook is quite easy. You need to enter a description in Markdown format and set up the Runbook steps.

AWS Creating Runbook

Again, it contains a lot of detail. For detailed information on how to create a Runbook in AWS, you can check the AWS documentation.

Finally, you can tag the Response plan you created if you want. After all these steps, you can create the Response plan with the “Create response plan” button. You can also edit and delete the Response Plan, Escalation Plan and Contacts you created later.

Starting A Incident

Now that we have done our preliminary preparation, we can return to the Incident Manager dashboard and start a new Incident. There are three ways we can start an Incident. These are:

In this article, I will talk about how to start an Incident manually and automaticly via CloudWatch.

Manually Create Incidents

Starting an Incident manually is quite easy and does not require much information. Click the “Start Incident” button on the Incident Manager dashboard. Select the Response Plan we prepared before and optionally give a title to the Incident and determine its Impact. Immediately afterward, the Incident is started by clicking the “Start” button.

AWS Incident Manager - Manually Incident Starting

Automatically Start Incidents With CloudWatch Alarms

With CloudWatch, we can track metrics and ensure that cases are automatically created in line with the conditions we want. For this, you must first create an Alarm from the CloudWatch panel. When creating an alarm, select Create Incident under Systems Manager Action menu and then select Response Plan to create an alarm.

AWS CloudWatch Incident Starting

From this moment on, an Incident will automatically occur in every situation that matches the condition you set when creating the Alarm.

Tracking And Resolving Incidents

You can follow the created cases from the Incident Manager dashboard. Here you can see general data about the cases, metrics, timeline, runbooks, engagements and you can edit some of them.

AWS Incident Manager - Incident Dashboad

In order to Resolve the Incident, all you need to do is to click on the “Resolve incident” button on the top left.

Post-Incident Analysis

After the relevant case is resolved, you can start an analysis of this case and review the issues related to improving your processes. This analysis process is done with Templates. You can create your Template or use the Template created by AWS. Analysis details include metrics, timeline, question set, actions and a checklist. Once an analysis has been created, some areas, such as the question set, can be edited later.

AWS Incident Manager - Incident Analysis

Isolating EC2 Instances

In the event of an incident on an EC2 Instance, it is critical to isolate that Instance from the network. The steps to be followed at this point are as follows:

I mentioned that I will not go into the details of Incident Response processes or AWS in this document, but the steps described (such as creating a new Security Group) are quite simple. If there are any missing points here, you can use AWS’s documents.

Open Source Incident Response Toolkit

There are some open open-source toolkits created by Andrew Krug, Alex McCormack, Joel Ferrier, and Jeff Parr that streamline our Incident Response processes in cloud environments.

Margarita Shotgun

Margarita Shotgun is a very simple to use memory dump tool written in Python programming language. It is designed to work in AWS environments. Detailed information is available in its own documentation.

Incident Pony

Incident Pony is a first of its kind case management and Incident Response orchestration tool specifically designed for AWS (Source: ThreatResponse).


AWS_IR CLI is the third and final Incident Response tool written by the ThreatResponse team. The purpose of the tool is to automate Incident Response processes. You can review the Black Hat document about the AWS_IR tool and other tools.

Yet Another Automation

There are many ways to automate Incident Response processes in an AWS environment. These processes can be automated with third-party tools and/or AWS’s own services. At this point, it is important to choose the most suitable solution for your needs. In this blogpost, I will show you an automation that AWS describes in their Security Blog and I will also link to some other automations.

The automatization we will use takes the necessary actions by following AWS GuardDuty and AWS Config controls. The architecture is as in the image below.

AWS Automated Incident Response Flowchart

The installation steps are quite easy. Because we don’t do the installation manually. We can quickly install it with CloudFormation Stack. CloudFormation Stack

In the Stack Parameters section, it asks us for some information. These are as follows:

AWS Automated Incident Response Stack Options

After all these settings, you can start Stack. Once Stack is complete, you will now have an automated IR process. For more details, you can read the AWS Security Blog.

Other Automations

I mentioned that there are many different ways to automate Incident Response processes in AWS. At this point, you should determine the most suitable solution for you. I have linked some other automatizations in the list below.

Last Word

In this blogpost, I aimed to give you basic information about Indicent Response processes in AWS. I hope it has contributed. Please note that Incident Response processes are not limited to what is described in this document. Incident Response and DFIR processes require expertise on their own. Also make sure to do the following in Incident Response processes in AWS:

If you have any suggestions for the article, please feel free to contact me through any communication channel (Linkedin, Twitter, Threema, etc.). I am constantly updating the articles in line with your feedback.