AWS科普系列:使用CloudTrail

2017-10-26  本文已影响336人  阿呆少爷

审计(Audit)是所有平台都需要的一个非常重要的基础功能。云计算平台的租户众多,需要审计每个租户的操作。AWS CloudTrail就是用以审计租户行为的一款产品。

从CloudTrail的Release Notes可以看出,这款产品2013-11-13发布了第一个版本,历史非常悠久,发展到今天,几乎所有的云产品都支持CloudTrail。生产者众多,消费者也不少,CloudTrail可以跟S3、CloudWatch和Macie打通,为用户分析审计数据提供良好的支持。下面一一盘点一下CloudTrail的主要特性和功能。

Your event history contains the create, modify, and delete activities for supported services taken by
people, groups, or AWS services in your AWS account. To view a complete log of your CloudTrail
events, create a trail and then go to your Amazon S3 bucket or CloudWatch Logs.

根据我的测试,七天事件并不会收集AssumeRole接口的信息,可能AWS觉得这是一个读操作吧。

$ aws sts assume-role --role-arn arn:aws:iam::978343370577:role/YQ001 --role-session-name henshao_test 
{
    "AssumedRoleUser": {
        "AssumedRoleId": "AROAJM25OSMMTTRM3B6ZS:henshao_test", 
        "Arn": "arn:aws:sts::978343370577:assumed-role/YQ001/henshao_test"
    }, 
    "Credentials": {
        "SecretAccessKey": "leSTW6PYPqV3s6Ut7/pSqq8N8EmcMI1qS0Gk+9ED", 
        "SessionToken": "FQoDYXdzEIf//////////wEaDCCm1qu+F6fmVTpbZiLwAQKu/hnQZ+kOVtZk9tyv/Ly406iNIMlMEF8naS9kzRWtq5eJHRV+hl8Uk+S0w3zTX7sq28CJQgS6MMVCGgRRweKgcT3370js+hYldyTykuVLTo5vNw2FKgK47kXv+eGZ3AS85LJbodk+WKTUx2jMLjyofL1G4h7pzm8L5pd3eqi+Wf7NElcETD7Pka9EMAC4pw7zEqECnNCAqUjoOKx4OT5s/VsdLFdGIShcC+jW+y7mAJBbObJbEoivK7N+5icG5KYoTZl9EEKm/hoh/8DxA+SwjKXJ0uc3bSTD1h8zc9YYqArAg1RNz3b4eqQb6PkRkyiql4XQBQ==", 
        "Expiration": "2017-11-07T06:59:06Z", 
        "AccessKeyId": "ASIAJYMPMMFBKWSSRTBQ"
    }
}

$ sleep 300 && aws cloudtrail lookup-events --lookup-attributes AttributeKey=EventName,AttributeValue=AssumeRole
{
    "Events": []
}

为了进一步验证这个结论。我分别创建一个Management Event All和Write-only的trail,使用cli做AssumeRole之后,将S3 Bucket同步到本地查了一下,只有All的trail才会有AssumeRole事件,Write-only的不会有。另外一点要注意的是,STS作为一个Global Service,它在每个region也有自己的endpoint。如果用户没有显式去deactive,那么会优先使用region的endpoint,这样日志也就记到了各个region里面了。详细信息请查看文档:Activating and Deactivating AWS STS in an AWS Region

$ zgrep -r -l AssumeRole yq001-all yq001-write-only 
yq001-all/.swp
yq001-all/AWSLogs/978343370577/CloudTrail/us-east-1/2017/11/08/978343370577_CloudTrail_us-east-1_20171108T0245Z_g9di7CwMhfzOIbRA.json.gz
yq001-all/AWSLogs/978343370577/CloudTrail/us-west-2/2017/11/08/978343370577_CloudTrail_us-west-2_20171108T0235Z_e1E521h8mjVNkrYi.json.gz
yq001-all/AWSLogs/978343370577/CloudTrail/us-west-2/2017/11/08/978343370577_CloudTrail_us-west-2_20171108T0240Z_E0X7iric1I1qrM1q.json.gz
yq001-all/AWSLogs/978343370577/CloudTrail/us-west-2/2017/11/08/978343370577_CloudTrail_us-west-2_20171108T0245Z_r4nygsE720Iwzb2q.json.gz
yq001-all/AWSLogs/978343370577/CloudTrail/us-west-2/2017/11/08/978343370577_CloudTrail_us-west-2_20171108T0250Z_nxVTzr1IAjQCOcNa.json.gz
yq001-all/AWSLogs/978343370577/CloudTrail/us-west-2/2017/11/08/978343370577_CloudTrail_us-west-2_20171108T0255Z_i680lZ9QzaD0sOKd.json.gz
$ aws cloudtrail get-event-selectors --trail-name yq0001
{
    "EventSelectors": [
        {
            "IncludeManagementEvents": true, 
            "DataResources": [
                {
                    "Values": [
                        "arn:aws:s3"
                    ], 
                    "Type": "AWS::S3::Object"
                }, 
                {
                    "Values": [
                        "arn:aws:lambda:ap-southeast-1:978343370577:function:CQLambda"
                    ], 
                    "Type": "AWS::Lambda::Function"
                }
            ], 
            "ReadWriteType": "All"
        }
    ], 
    "TrailARN": "arn:aws:cloudtrail:us-east-1:978343370577:trail/yq0001"
}

每个trail有一个HomeRegion属性,用于标识这个trail是哪个region创建的。AWS CLI默认会把shadow trail显示出来。需要带上--no-include-shadow-trails参数才不会显示shadow trail。

$ aws cloudtrail describe-trails               
{
    "trailList": [
        {
            "IncludeGlobalServiceEvents": true, 
            "Name": "test001", 
            "TrailARN": "arn:aws:cloudtrail:us-east-2:978343370577:trail/test001", 
            "LogFileValidationEnabled": true, 
            "IsMultiRegionTrail": true, 
            "HasCustomEventSelectors": false, 
            "S3BucketName": "cloudtrail-yq002-s3", 
            "HomeRegion": "us-east-2"
        }, 
        {
            "IncludeGlobalServiceEvents": true, 
            "Name": "test001", 
            "TrailARN": "arn:aws:cloudtrail:us-east-1:978343370577:trail/test001", 
            "LogFileValidationEnabled": false, 
            "IsMultiRegionTrail": false, 
            "HasCustomEventSelectors": false, 
            "S3BucketName": "cloudtrail-yq002-s3", 
            "HomeRegion": "us-east-1"
        }
    ]
}

比如我在us-east-2(Ohio)创建了foo这个trail,切换到us-east-1(Virginia)也能看到这个trail,但是如果要修改这个trail,必须切换到us-east-2(Ohio)才行。

image.png

接着在IAM里面创建一个名为hs001的子账号,然后去这两个trail的S3 bucket里面查看日志。可以发现这两个bucket里面包含了相同的日志文件。

该事件具体信息如下所示。

{
    "eventVersion": "1.02",
    "userIdentity": {
        "type": "Root",
        "principalId": "978343370577",
        "arn": "arn:aws:iam::978343370577:root",
        "accountId": "978343370577",
        "accessKeyId": "ASIAJYGQDQLTAMBTNF4A",
        "sessionContext": {
            "attributes": {
                "mfaAuthenticated": "false",
                "creationDate": "2017-10-25T17:29:43Z"
            }
        }
    },
    "eventTime": "2017-10-26T02:56:57Z",
    "eventSource": "iam.amazonaws.com",
    "eventName": "CreateUser",
    "awsRegion": "us-east-1",
    "sourceIPAddress": "42.120.74.107",
    "userAgent": "console.amazonaws.com",
    "requestParameters": {
        "userName": "hs001"
    },
    "responseElements": {
        "user": {
            "path": "/",
            "arn": "arn:aws:iam::978343370577:user/hs001",
            "userId": "AIDAJW6ORA5TRHZCPZYWC",
            "createDate": "Oct 26, 2017 2:56:57 AM",
            "userName": "hs001"
        }
    },
    "requestID": "54c485da-b9f9-11e7-aa4f-633c5c77f490",
    "eventID": "32cec936-d4e1-4ed9-9bc7-0f673b262c0c",
    "eventType": "AwsApiCall",
    "recipientAccountId": "978343370577"
}
#通过AWS CLI将bucket同步到本地,然后使用zgrep去查找相应的事件。
$ aws s3 sync s3://cloudtrail-cq001-s3 cloudtrail-cq001-s3 
$ aws s3 sync s3://cloudtrail-yq003-s3 cloudtrail-yq003-s3 

$ zgrep -r -l CreateUser cloudtrail-cq001-s3/*  cloudtrail-yq003-s3/*
cloudtrail-cq001-s3/AWSLogs/978343370577/CloudTrail/us-east-1/2017/10/26/978343370577_CloudTrail_us-east-1_20171026T0300Z_P8mz77tfnWK5AWqp.json.gz
cloudtrail-yq003-s3/AWSLogs/978343370577/CloudTrail/us-east-1/2017/10/26/978343370577_CloudTrail_us-east-1_20171026T0300Z_2Mccc89EFYGv0QB9.json.gz

$ md5 cloudtrail-cq001-s3/AWSLogs/978343370577/CloudTrail/us-east-1/2017/10/26/978343370577_CloudTrail_us-east-1_20171026T0300Z_P8mz77tfnWK5AWqp.json.gz cloudtrail-yq003-s3/AWSLogs/978343370577/CloudTrail/us-east-1/2017/10/26/978343370577_CloudTrail_us-east-1_20171026T0300Z_2Mccc89EFYGv0QB9.json.gz
MD5 (cloudtrail-cq001-s3/AWSLogs/978343370577/CloudTrail/us-east-1/2017/10/26/978343370577_CloudTrail_us-east-1_20171026T0300Z_P8mz77tfnWK5AWqp.json.gz) = a76c3f4910953ff0b3ce15b5c5871e25
MD5 (cloudtrail-yq003-s3/AWSLogs/978343370577/CloudTrail/us-east-1/2017/10/26/978343370577_CloudTrail_us-east-1_20171026T0300Z_2Mccc89EFYGv0QB9.json.gz) = a76c3f4910953ff0b3ce15b5c5871e25

示意图如下所示。

image.png

Logging IAM Events with AWS CloudTrail文档可以看出STS global endpoint的日志是要记录到us-east-1的,但是文档中没有提到IAM的endpoint:https://iam.amazonaws.com 日志会记录到哪个region,应该也是us-east-1

另外七天事件是trail的数据源,每个region的七天事件都包含了全局服务的事件,所以这个地方是有冗余的。结合起来的示意图如下所示。

image.png

从lookup-events接口也可以看出来,切换到不同的region,能搜索到同样的全局的事件。

image.png
$ aws cloudtrail create-trail --name test001 --s3-bucket-name cloudtrail-yq002-s3

An error occurred (TrailAlreadyExistsException) when calling the CreateTrail operation: Trail test001 already exists for customer: 978343370577

#接口默认会获取本region是Home Region的trail的信息
$ export AWS_DEFAULT_REGION='us-east-1'; aws cloudtrail get-trail-status --name test001
{
    "LatestDeliveryAttemptTime": "", 
    "LatestNotificationAttemptSucceeded": "", 
    "LatestDeliveryAttemptSucceeded": "", 
    "IsLogging": false, 
    "TimeLoggingStarted": "", 
    "LatestNotificationAttemptTime": "", 
    "TimeLoggingStopped": ""
}

$ export AWS_DEFAULT_REGION='us-east-2'; aws cloudtrail get-trail-status --name test001
{
    "LatestNotificationAttemptSucceeded": "", 
    "LatestDeliveryAttemptTime": "2017-10-26T08:23:28Z", 
    "LatestDeliveryTime": 1509006208.65, 
    "LatestDeliveryAttemptSucceeded": "2017-10-26T08:23:28Z", 
    "IsLogging": true,
    "TimeLoggingStarted": "2017-10-26T03:27:07Z", 
    "StartLoggingTime": 1508988427.631, 
    "LatestDigestDeliveryTime": 1509021950.222, 
    "LatestNotificationAttemptTime": "", 
    "TimeLoggingStopped": ""
}

#对于shadow trail,会提示不存在。
$ export AWS_DEFAULT_REGION='us-west-2'; aws cloudtrail get-trail-status --name test001

An error occurred (TrailNotFoundException) when calling the GetTrailStatus operation: Unknown trail: test001 for the user: 978343370577
image.png image.png image.png

CloudTrail跟CloudWatch结合起来之后,就可以利用CloudWatch更强大的Filter and Pattern Syntax分析日志。

image.png

使用CloudWatch还可以针对CloudTrail收集到的事件创建Metric和Alarm,这样做监控报警也是非常方便的。CloudTrail跟CloudWatch Events结合起来就更加强大了,可以通过事件触发执行自定义的Lambda函数,做更加丰富和复杂的事情。参考文章:Creating CloudWatch Alarms for CloudTrail Events: ExamplesCreating a CloudWatch Events Rule That Triggers on an AWS API Call Using AWS CloudTrail

image.png image.png

参考资料。

  1. Limits in AWS CloudTrail
上一篇 下一篇

猜你喜欢

热点阅读