In this post we are going to look at how to enable and use access logs for API Gateway in CloudWatch.

Just a quick recap, there are two ways of logging API Gateway:

  • Execution logs: Logs with detailed information as API Gateway goes through each step of processing the request. Useful for tracing individual requests. Can generate lots of log data, resulting in a large CloudWatch bill.
  • Access logs: Logs of who has accessed your API. Each request generates a single entry in the logs, similar to NGINX logs. Useful for sending to an analytics tool to gather metrics.

We have a detailed post looking at the differences between execution logs and access logs here.

Seed has built-in support for API Gateway access logs. It can turn it on for you in one click and you can even view it directly in the Seed console. You can read more on it here in our docs.

Let’s start by looking at how to enable access logs.

Enabling API Gateway access logs

This is a two step process. First, we need to create an IAM role that allows API Gateway to write logs to CloudWatch. Then we need to turn on logging for our API Gateway project.

Start by logging into your AWS Console and select IAM from the list of services.

Select IAM from list of AWS services

Click Roles on the left menu.

Select IAM Roles from left menu

Click Create role.

Select Create IAM Role

Under AWS service, select API Gateway.

Select API Gateway from AWS services

Click Next: Permissions.

Click Next: Permissions

Click Next: Review.

Click Next: Review

Enter a Role name and click Create role. In our case, we call our role APIGatewayCloudWatchLogs.

Enter a role name and create role

Click on the role we just created.

Click on new role

Make a note of the Role ARN. We’ll be needing this soon.

Copy Role ARN

Now that we’ve created an IAM role, let’s turn on logging for our API Gateway project.

Go back to your AWS Console and select API Gateway from the list of services.

Select API Gateway from list

Click on Settings in the left panel.

Click on API Gateway settings

Enter the ARN of the IAM role we just created in the CloudWatch log role ARN field and hit Save.

Enter IAM Role ARN and save

Select your API project from the left panel, click Stages, then pick the stage you want to enable logging for. For our API, we deployed it to the prod stage.

Select API Gateway project and stage

In the Logs tab:

  • Check Enable Access Logging.
  • Enter a CloudWatch Group name with the API Gateway id and stage name to ensure uniqueness. Ie, API-Gateway-Access-Logs_{API_GATEWAY_ID}/{STAGE}.
  • Enter the Log Format or pick one of the predefined log format in CLF, JSON, XML or CSV. 
    {
  "requestId":"$context.requestId",
  "ip": "$context.identity.sourceIp",
  "caller":"$context.identity.caller",
  "user":"$context.identity.user",
  "requestTime":"$context.requestTime",
  "httpMethod":"$context.httpMethod",
  "resourcePath":"$context.resourcePath",
  "status":"$context.status",
  "protocol":"$context.protocol",
  "responseLength":"$context.responseLength"
}

Enable Access Logging and set log format

Scroll to the bottom of the page and click Save changes. Now our API Gateway requests should be logged via CloudWatch.

Viewing API Gateway access logs

CloudWatch groups log entries into Log Groups and then further into Log Streams. Log Groups and Log Streams can mean different things for different AWS services. For API Gateway, when logging is first enabled in an API project’s stage, API Gateway creates 1 log group for the stage, and 300 log streams in the group ready to store log entries. API Gateway picks one of these streams when there is an incoming request.

To view API Gateway logs, log in to your AWS Console and select CloudWatch from the list of services.

Select CloudWatch service

Select Logs from the left panel.

Select CloudWatch logs

Select the log group that starts with API-Gateway-Access-Logs_ followed by the API Gateway id.

Select log group with API Gateway id

You should see 300 log streams ordered by the last event time. This is the last time a request was recorded. Click on the first stream.

Select first log stream from group

This shows you one log entry for each API request. Expand a row, the log data should reflect the format you had previously defined.

Select log row from API Gateway request

Note that, two consecutive groups of logs are not necessarily two consecutive requests in real time. This is because there might be other requests that are processed in between these two that were picked up by one of the other log streams.

Summary

This post should give you a good idea of how to enable access logs for your API Gateway project and also how to view them from the CloudWatch console.