In this post we are going to look at how to enable and use execution logs for API Gateway in CloudWatch.
Just a quick recap, there are two ways of logging API Gateway:
- Execution logs: Logs with detailed information as API Gateway goes through each step of processing the request. Useful for tracing individual requests. Can generate lots of log data, resulting in a large CloudWatch bill.
- Access logs: Logs of who has accessed your API. Each request generates a single entry in the logs, similar to NGINX logs. Useful for sending to an analytics tool to gather metrics.
We have a detailed post looking at the differences between execution logs and access logs here.
Let’s start by looking at how to enable execution logs.
Enabling API Gateway execution logs
This is a two step process. First, we need to create an IAM role that allows API Gateway to write logs to CloudWatch. Then we need to turn on logging for our API Gateway project.
Start by logging into your AWS Console and select IAM from the list of services.
Click Roles on the left menu.
Click Create role.
Under AWS service, select API Gateway.
Click Next: Permissions.
Click Next: Review.
Enter a Role name and click Create role. In our case, we call our role
Click on the role we just created.
Make a note of the Role ARN. We’ll be needing this soon.
Now that we’ve created an IAM role, let’s turn on logging for our API Gateway project.
Go back to your AWS Console and select API Gateway from the list of services.
Click on Settings in the left panel.
Enter the ARN of the IAM role we just created in the CloudWatch log role ARN field and hit Save.
Select your API project from the left panel, click Stages, then pick the stage you want to enable logging for. For our API, we deployed it to the prod stage.
In the Logs tab:
- Check Enable CloudWatch Logs.
- Select INFO for Log level to log every request.
- Check Log full requests/responses data to include entire request and response body in the log.
- Check Enable Detailed CloudWatch Metrics to track latencies and errors in CloudWatch metrics.
Scroll to the bottom of the page and click Save changes. Now our API Gateway requests should be logged via CloudWatch.
Viewing API Gateway execution logs
CloudWatch groups log entries into Log Groups and then further into Log Streams. Log Groups and Log Streams can mean different things for different AWS services. For API Gateway, when logging is first enabled in an API project’s stage, API Gateway creates 1 log group for the stage, and 300 log streams in the group ready to store log entries. API Gateway picks one of these streams when there is an incoming request.
To view API Gateway logs, log in to your AWS Console and select CloudWatch from the list of services.
Select Logs from the left panel.
Select the log group that starts with
API-Gateway-Execution-Logs_ followed by the API Gateway id.
You should see 300 log streams ordered by the last event time. This is the last time a request was recorded. Click on the first stream.
This shows you one log entry for each API request. Expand a row, the log data should reflect the format you had previously defined.
Note that, two consecutive groups of logs are not necessarily two consecutive requests in real time. This is because there might be other requests that are processed in between these two that were picked up by one of the other log streams.
This post should give you a good idea of how to enable execution logs for your API Gateway project and also how to view them from the CloudWatch console.