In this blog post, we will see how to analyze S3 logs using Athena.
S3 Access Logs :
You can identify Amazon S3 requests with Amazon S3 access logs. Server access logging provides detailed records for the requests that are made to an Amazon S3 bucket. Server access logs are useful for many applications. For example, access log information can be useful in security and access audits.
Step 1 – We have login AWS console account using IAM user and password.
Step 2 – Go to S3 service and select your S3 bucket.
Step 3 – Create logging buckets in the following Regions.
Step 4 – Go to S3 bucket Properties tab and select Logging.
Step 5 – Enable the Logging and give the Target logging bucket name and Save.
Step 6 – Go to Athena service
Step 7 – Now create database and run query. In the Query Editor, run a command similar to the following:
create database s3_access_logs_db.
Step 8 – In the Query Editor, run a command similar to the following to create a table schema in the database.
Step 9 – In the navigation pane, under Database, choose your database.
Step 10 – Under Tables, choose Preview table next to your table name.
Step 11 – In the Results pane, you should see data from the server access logs, such as bucketowner, bucket, requestdatetime, and so on.
Example — show who deleted an object and when (timestamp, IP address, and IAM user)
SELECT RequestDateTime, RemoteIP, Requester, Key
FROM s3_access_logs_db.mybucket_logs
WHERE key = ‘images/picture.jpg’ AND operation like ‘%DELETE%’;
Example — show all operations executed by an IAM user.
SELECT *
FROM s3_access_logs_db.mybucket_logs
WHERE requester=’arn:aws:iam::123456789123:user/user_name’;
