AWS Config & Conformance Packs

Introduction

With ever-increasing cloud adoption, its associated usage across Businesses, and the never-ending complex sprawl of resources, ensuring compliance is more critical than ever to maintaining security, integrity, and operational hygiene. Tag compliance ensures that all resources adhere to predefined standards, helping organizations track ownership, purpose, and lifecycle of resources while also supporting reporting and auditing needs.

AWS Config and Conformance Packs offer a powerful framework for managing and enforcing configuration standards at scale. By leveraging these tools, organizations can define, track, and remediate their infrastructure resources as code. This approach enables continuous monitoring of resource compliance, aligning infrastructure with governance policies, and automating corrective actions to ensure consistency across AWS environments. In this blog, we’ll explore how to implement this approach using AWS Config and Conformance Packs to achieve tag compliance across your AWS organization.

The best-case scenario

Before we delve into implementing something that ensures tag compliance reactively, it’s important to understand that the ideal scenario is to establish well-defined tagging strategies early on, typically during the Landing Zone design and initial implementation. This includes enforcing mandatory tags and ensuring that no resources are created without essential tags, alongside “should-have” tags that allow tracking and improving tagging coverage over time.

However, in reality, many organizations are scaling rapidly in the cloud — often driven by priorities like data centre exits — leaving little room for proactive tagging governance. As a result, they are left with a lack of visibility and control that hinders their ability to manage resources efficiently. We will go through remediating such “complex and difficult to manage” use cases from a tagging perspective.

The Framework

The tagging framework outlined here leverages AWS Config, Conformance Packs, AWS Systems Manager Runbooks, and AWS Organizations. This framework enables the following key capabilities –

• Centralized control of tag compliance from the management or delegated administrator account.
• Flexibility in targeting specific accounts for compliance, ensuring an ability to implement a phased approach.
• The ability to define which types of resources are subject to tag compliance, tailoring the scope to the needs.
• A solution for tagging resources not natively supported by AWS Config’s required-tags managed rule, using customizations within AWS Systems Manager Runbooks.

Services used in the solution

• AWS Config provides a detailed view of the configuration of AWS resources in the AWS account. This includes how the resources are related to one another and how they were configured in the past, allowing us to see how the configurations and relationships change over time.
• AWS Systems Manager is a secure end-to-end management solution for hybrid cloud environments. We will use it to remediate non-compliant resources automatically using an Automation runbook. This will take inputs from the AWS config and include sample customisation for a resource type not supported by the required-tags managed rule.
• AWS Organizations lets us create new AWS accounts at no additional charge. With accounts in an organization, we can easily allocate resources, group accounts, and apply governance policies to accounts or groups.
• AWS CloudFormation is used to automate the tagging framework.

Solution Overview

The solution revolves around the capability of AWS Systems Manager, which automatically remediates non-compliant resources evaluated by the AWS Config rule required-tags. The solution’s focus is on ensuring that accounts can be remediated in small to large AWS organizations with multiple accounts and their varying needs around tax compliance.

High-Level Process

On a high level, the steps mentioned below enable the framework in the AWS organization.

Prerequisites

• AWS Configuration must be enabled in all the accounts in the AWS organization. (If not enabled, it can be done using AWS config recording Quick Setup, a capability of AWS Systems Manager that allows the quick creation of a configuration recorder.
• AWS Organization delegated administrator account ID (or management account ID).
• Applicable Tag values are created as the JSON file (tags.json), like below (replacing the appropriate accountNo and adding appropriate tags)

[
    {
        "accountNo": "XXXXXXXXXXXXX",
        "Ops:ApplicationName": "ABC-DEF",
        "Bus:BusinessOwner": "abc.xyz",
        "Environment": "QA"
     },
    {
        "accountNo": "YYYYYYYYYYYY",
        "Ops:ApplicationName": "XYZ-DEF",
        "Bus:BusinessOwner": "def.xyz",
        "Environment": "Test",
    }]

• A list of member account IDs to be excluded from the deployment.
• A list of member account IDs where the system manager document will be shared.
• The S3 bucket in the management / delegated administrator account will hold the conformance pack Cloud formation template and tags.json file.
• The S3 bucket should also have the following bucket policy deployed (replace ACCOUNT_NUMBER, BUCKET_NAME, & ORG_ID in the policy)

{
    "Version": "2012-10-17",
    "Id": "BucketPolicy",
    "Statement": [
        {
            "Sid": "GrantFullAccessToRootUser",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<ACCOUNT_NUMBER>:root"
            },
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::<BUCKET_NAME>",
                "arn:aws:s3:::<BUCKET_NAME>/*"
            ]
        },
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:Get*",
            "Resource": [
                "arn:aws:s3:::<BUCKET_NAME>",
                "arn:aws:s3:::<BUCKET_NAME>/*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:PrincipalOrgID": "<ORG_ID>"
                }
            }
        },
        {
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:List*",
            "Resource": [
                "arn:aws:s3:::<BUCKET_NAME>",
                "arn:aws:s3:::<BUCKET_NAME>/*"
            ],
            "Condition": {
                "StringEquals": {
                    "aws:PrincipalOrgID": "<ORG_ID>"
                }
            }
        }
    ]
}

Detailed Process

IAM Role Deployment

This step guides you to which IAM role is required to invoke the remediation action that will be created subsequently as part of the Create Automation Runbook section.

The IAM cloud formation template is below. Please note that permissions are based on what’s supported by AWS Config’s required-tags capabilities and the custom actions defined in the runbook for ECS service only, which isn’t supported by required-tags functionality. (For a full set of resources and an IAM role, visit the GitHub link at the bottom of this page.)

AWSTemplateFormatVersion: 2010-09-09
Description: This CloudFormation template creates IAM role which can be assumed by the SSM runbook with the permission to tag resources
Resources:
  TagRemediationAutomationRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - ssm.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Description: role for tag remediation automation
      RoleName: TagRemediationAutomationRole

  TagRemediationAutomationPolicy:
    Type: 'AWS::IAM::Policy'
    Properties:
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Action:
              - 'config:GetComplianceDetailsByConfigRule'
              - 'config:DescribeConformancePacks'
              - 'config:BatchGetResourceConfig'
              - 'tag:GetResources'
              - 'tag:GetTagKeys'
              - 'tag:GetTagValues'
              - 'tag:TagResources'
            Resource: '*'
          - Effect: Allow
            Action:
              - 'ec2:CreateTags' 
              - 's3:PutBucketTagging'
              - 's3:GetBucketTagging'
              - "ecs:ListClusters"
              - "ecs:ListServices"
              - "ecs:ListTasks"
              - "ecs:ListContainerInstances"
              - "ecs:DescribeClusters"
              - "ecs:DescribeServices"
              - "ecs:DescribeTasks"
              - "ecs:DescribeContainerInstances"
              - 'ecs:TagResource'
            Resource: '*'
      PolicyName: TagRemediationAutomationPolicy
      Roles:
        - !Ref TagRemediationAutomationRole

Outputs:
  RoleName:
    Description: Name of Created IAM Role
    Value: !Ref TagRemediationAutomationRole
  RoleArn:
    Description: Arn of Created Role
    Value: !GetAtt TagRemediationAutomationRole.Arn

• Deploy the Cloud Formation StackSet from the management / delegated administrator account using the template above.
• Set deployment options, such as regions and specific accounts, where you want to deploy.
• Deploy the Cloud Formation Stack from the management / delegated administrator account using the template above.

When the IAM role deployment is completed, we have the role that can be assumed by the Automation runbook triggered by the remediation action defined in AWS Config.

Automation Runbook Deployment

Remediation of non-compliant resources evaluated by the AWS Config rule is performed using Automation runbooks. To maintain centrally managed runbooks throughout the organization, this runbook is created using a CloudFormation stack in the management account / delegated administrator account and shared with the member accounts through the management console.

Automation Runbooks looks like below –

AWSTemplateFormatVersion: 2010-09-09
Description: This CloudFormation template creates the SSM runbook that remediates the non-compliant resource checked by the required-tags managed rule.
Resources:
  SsmDocumentTagRemediation:
    Type: AWS::SSM::Document
    Properties:
      Content:
        description: |
          ### Document Name - SSM-tag-remediation-automation

          ## What does this document do?
          This document adds the specified tags to the resource checked by required-tags managed rule.

          ## Input Parameters
          * TagJson: (Required) JSON array of objects with account-specific tags.
          * AutomationAssumeRole: (Optional) This document uses the role specified in assumeRole parameter, will not use AutomationAssumeRole.
          * ConformancePackName: Used to find the detail info of the non-compliant resource.

          ## Output Parameters
          * SetRequiredTags.SuccessfulResources: A list of the resources that were successfully tagged.
          * SetRequiredTags.FailedResources: A mapList of the resources where tagging failed and the reason for the failure.
        schemaVersion: '0.3'
        assumeRole: 'arn:{{global:AWS_PARTITION}}:iam::{{global:ACCOUNT_ID}}:role/TagRemediationAutomationRole'
        outputs:
          - SetRequiredTags.SuccessfulResources
          - SetRequiredTags.FailedResources
        parameters:
          RequiredTags:
            type: StringMap
            description: (Required) The tags to add to the resources.
            displayType: textarea
          TagJson:
            type: String
            description: (Required) The JSON array of account-specific tags.
            displayType: textarea
          S3Bucket:
            type: String
            description: (Required) The S3 Bucket where tag values will be stored.
            displayType: textarea  
          ResourceID:
            type: String
            description: (Required) Non-compliant item ResourceID.
            displayType: textarea
          AccountID: 
            type: String
            description: The AWS Account ID.
            displayType: textarea
          ConformancePackName:
            type: String
            description: (Required) Conformance Pack name passed to the automation.
            displayType: textarea
          AutomationAssumeRole:
            type: String
            description: (Optional) The ARN of the role that allows Automation to perform the actions on your behalf.
            default: ''
            allowedPattern: '^arn:aws(-cn|-us-gov)?:iam::\d{12}:role/[\w+=,.@_\/-]+|^$'
        mainSteps:
          - name: SetRequiredTags
            action: 'aws:executeScript'
            description: |
              ## SetRequiredTags
              Adds the specified tags to non-compliant resources checked by the required-tags managed rule.
              ## Inputs
              * TagJson: (String) The JSON array of account-specific tags.
              ## Outputs
              * SuccessfulResources: A list of the resources that were successfully tagged.
              * FailedResources: A mapList of the resources where tagging failed and the reason for the failure.
            onFailure: Abort
            isCritical: true
            timeoutSeconds: 600
            isEnd: true
            inputs:
              Runtime: python3.11
              Handler: set_required_tags_handler
              InputPayload:
                RequiredTags: '{{RequiredTags}}'
                ResourceID: '{{ResourceID}}'
                ConformancePackName: '{{ConformancePackName}}'
                AccountID: '{{AccountID}}'
                TagJson: '{{TagJson}}'
                S3Bucket: '{{S3Bucket}}'
              Script: |
                import json
                import boto3

                def read_tag_values(s3Bucket, fileName): 
                  s3 = boto3.resource('s3')
                  obj = s3.Object(s3Bucket, fileName)
                  try: 
                    data = obj.get()['Body'].read()
                    data_str = data.decode('utf-8')
                    data_json = json.loads(data_str)
                    return data_json
                  except Exception as e:
                    return e

                def tag_ecs_resources(tags): 
                  client = boto3.client('ecs')
                  success = []
                  failed = []
                   try:
                    clusters = client.list_clusters()['clusterArns']
                    for cluster_arn in clusters: 
                      try:
                        success.append(cluster_arn)
                        services = client.list_services(cluster=cluster_arn)['serviceArns']
                        for service_arn in services:
                          try:
                            success.append(service_arn)
                          except Exception as e:
                            failed.append({'ResourceArn': service_arn, 'error': str(e)})
                        tasks = client.list_tasks(cluster=cluster_arn)['taskArns']
                        for task_arn in tasks:
                          try: 
                            success.append(task_arn)
                          except Exception as e:
                            failed.append({'ResourceArn': task_arn, 'error': str(e)})
                        container_instances = client.list_container_instances(cluster=cluster_arn)['containerInstanceArns']
                        for container_instance_arn in container_instances:
                          try:
                            success.append(container_instance_arn)    
                          except Exception as e:
                            failed.append({'ResourceArn': container_instance_arn, 'error': str(e)})
                      except Exception as e:
                        failed.append({'ResourceArn': cluster_arn, 'error': str(e)})
                     for resource_arn in success:
                      try:
                        client.tag_resource(resourceArn=resource_arn, tags=tags)
                       except Exception as e:
                        failed.append({'ResourceArn': resource_arn, 'error': str(e)})
                  except Exception as e:
                    failed.append({'ResourceArn': 'none', 'error': str(e)})

                  return {'SuccessResources': success, 'FailedResources': failed}  

                 def set_required_tags(resourcesArn, tags):
                    client = boto3.client('resourcegroupstaggingapi')
                    successResources = []
                    failedResources = []

                    for arn in resourcesArn:
                        try:
                            response = client.tag_resources(ResourceARNList=[arn], Tags=tags)
                            successResources.append(arn)
                        except Exception as e:
                            errorMsg = str(e)
                            failedResources.append({'ResourceArn': arn, 'error': errorMsg})

                    return {'SuccessResources': successResources, 'FailedResources': failedResources}

                def get_config_rule_name(prefix):
                    client = boto3.client('config')
                    ConformancePackId = ''
                    config_rule_name = ''

                    try:
                        response = client.describe_conformance_packs()
                        for cp in response['ConformancePackDetails']:
                            if prefix+'-' in cp['ConformancePackName']:
                                ConformancePackId = cp['ConformancePackId']
                                break
                    except Exception as e:
                        errorMsg = str(e)
                        print(errorMsg)

                    config_rule_name = 'required-tags-' + ConformancePackId
                    return config_rule_name

                def set_required_tags_handler(event, context):
                    print(event)

                    ResourceID = event['ResourceID']
                    TagJson = event['TagJson']
                    ConformancePackName = event['ConformancePackName']
                    AccountID = event['AccountID']
                    s3Bucket = event['S3Bucket']

                    # Parse TagJson
                    tag_json = json.loads(TagJson)
                    contents = read_tag_values(s3Bucket, 'tags.json')
                    if not contents:
                        raise Exception(f"No tags found for account ID {AccountID}")

                    print('printing contents from the main function - line 275')
                    print(contents)

                    # Find the tags for the current account
                    tags = next((x for x in contents if x['accountNo'] == AccountID), None)
                    if not tags:
                        raise Exception(f"No tags found for account ID {AccountID}")

                    resourcesArn = []
                    resource_type = None
                    config_client = boto3.client('config')

                    # Obtain the rule name to match the ResourceID
                    config_rule_name = get_config_rule_name(ConformancePackName)
                    print(config_rule_name)

                    # Query non-compliant resources in the rule to find the ResourceID match and getting the resource_type
                    try:
                        response = config_client.get_compliance_details_by_config_rule(
                            ConfigRuleName=config_rule_name,
                            ComplianceTypes=['NON_COMPLIANT'],
                            Limit=10
                        )
                    except Exception as e:
                        errorMsg = str(e)
                        print(errorMsg)

                    query_result = response['EvaluationResults']
                    for r in query_result:
                        if r['EvaluationResultIdentifier']['EvaluationResultQualifier']['ResourceId'] == ResourceID:
                            resource_type = r['EvaluationResultIdentifier']['EvaluationResultQualifier']['ResourceType']
                            resource_id = r['EvaluationResultIdentifier']['EvaluationResultQualifier']['ResourceId']
                            break

                    while 'NextToken' in response:
                        try:
                            response = config_client.get_compliance_details_by_config_rule(
                                ConfigRuleName=config_rule_name,
                                ComplianceTypes=['NON_COMPLIANT'],
                                Limit=10,
                                NextToken=response['NextToken']
                            )
                        except Exception as e:
                            errorMsg = str(e)
                            print(errorMsg)

                        query_result = response['EvaluationResults']
                        for r in query_result:
                            if r['EvaluationResultIdentifier']['EvaluationResultQualifier']['ResourceId'] == ResourceID:
                                resource_type = r['EvaluationResultIdentifier']['EvaluationResultQualifier']['ResourceType']
                                resource_id = r['EvaluationResultIdentifier']['EvaluationResultQualifier']['ResourceId']
                                break

                    if not resource_type:
                        raise Exception(f"Resource ID {ResourceID} not found in any non-compliant results.")

                    # Use the resource type and ResourceID to find the ARN
                    response = config_client.batch_get_resource_config(
                        resourceKeys=[
                            {
                                'resourceType': resource_type,
                                'resourceId': ResourceID
                            }
                        ]
                    )

                    resources = response.get('baseConfigurationItems', [])
                    if not resources:
                        raise Exception(f"No resource found with Resource ID {ResourceID} and type {resource_type}")
                    resourceArn = resources[0]['arn']
                    resourcesArn.append(resourceArn)

                    # Add tags based on resource type
                    out = []
                    out.append(set_required_tags(resourcesArn, tags))
                    formatted_tags = [{'key': k, 'value': v} for k, v in tags.items()]
                    out.append(tag_ecs_resources(formatted_tags))
                    return out
            outputs:
              - Name: SuccessfulResources
                Selector: $.Payload.SuccessResources
                Type: StringList
              - Name: FailedResources
                Selector: $.Payload.FailedResources
                Type: MapList
      DocumentFormat: YAML
      DocumentType: Automation
      Name: 'SsmDocumentTagRemediation'

Please note that the runbook includes a custom method to patch the ECS resources (as an example), which are not supported by the required tags functionality of AWS Config. Based on the available tagJson parameter, these resources will always tag the services irrespective of whether the tags are present on the services with the right values. This is to avoid reading and updating the tag information (consuming more API calls).

Once this template is deployed through Cloud Formation, it creates an automation runbook named SsmDocumentTagRemediation which uses the resourceID passed by AWS Config rules to query the ARNs of the non-compliant resources and then uses AWS resource group tagging API to tag those resources.

Share the runbook with the member accounts

The document is then shared with the member accounts, where we expect the tagging remediation and ongoing compliance to be run. To do this –

1. Open the AWS Systems Manager console in the management account, then choose Documents in the left navigation pane.
2. Choose the Owned by Me tab and select the SsmDocumentTagRemediation runbook created in the previous step.
3. Select the Details tab. In the permissions section, add any account with which you would like to share this runbook and choose Save to save the changes. (Please note: there is a quota that allows a single Systems Manager document to be shared with a maximum of 1000 AWS accounts.)

Deploy the Conformance Pack

The management/delegated administrator account deploys a conformance pack that contains the check and relevant remediation for tags.

The conformance pack looks like the following:

Parameters:
  ConformancePackName:
    Description: The name intented to use when creating the organization conformance pack.
    Default: 'TagRemediationConfPack'
    Type: String
  tag1Key:
    Description: Flag to denote that the tag is applied by the AWS Config and Conformance Pack.
    Type: String
    Default: TagAppliedByConfig
  tag1Value: 
    Description: Falg value to denote that the tag is applied by the AWS Config and Conformance Pack.
    Type: String
    Default: 'True'
  S3Bucket:
    Description: s3 Bucket where tag values are stored.
    Type: String
    Default: tstcontmpl
  TagJson: 
    Description: JSON string input for all the Tags.
    Type: String
    Default: 'test'
Resources:
  ResourceTaggingCheck:
    Type: AWS::Config::ConfigRule
    Properties:
      ConfigRuleName: required-tags
      Description: Check resources against the required tags
      Source:
        Owner: AWS
        SourceIdentifier: REQUIRED_TAGS
      InputParameters:
        tag1Key: !Ref tag1Key
        tag1Value: !Ref tag1Value
      Scope:
        ComplianceResourceTypes:
           - AWS::EC2::Instance
           - AWS::S3::Bucket

  MapTaggingRemediation:
    DependsOn: ResourceTaggingCheck
    Type: 'AWS::Config::RemediationConfiguration'
    Properties:
      ConfigRuleName: required-tags
      TargetId: !Sub 'arn:aws:ssm:<REGION>:<ACCOUNT>:document/SsmDocumentTagRemediation'
      TargetType: 'SSM_DOCUMENT'
      Parameters:
        RequiredTags:
          StaticValue:
            Values:
              - !Sub 
                - '{"${tag1Key}": "${tag1Value}"}'
                - tag1Key: !Ref tag1Key
                  tag1Value: !Ref tag1Value
        AccountID:
          StaticValue:
            Values:
              - !Sub '${AWS::AccountId}'          
        S3Bucket:
          StaticValue:
            Values:
              - Ref: S3Bucket
        TagJson: 
          StaticValue:
            Values:
              - Ref: TagJson
        ConformancePackName:
          StaticValue:
            Values:
              - Ref: ConformancePackName
        ResourceID:
          ResourceValue:
            Value: 'RESOURCE_ID'
        AutomationAssumeRole:
          StaticValue:
            Values:
              - 'arn:aws:iam::<ACCOUNT>:role/TagRemediationAutomationRole'
      ExecutionControls:
        SsmControls:
          ConcurrentExecutionRatePercentage: 10
          ErrorPercentage: 10
      Automatic: True
      MaximumAutomaticAttempts: 10
      RetryAttemptSeconds: 600

As more accounts are added to be remediated through the framework, modifications may be needed in the TagJson parameter to include the specific tags in each account. Please note that the CloudFormation template contains all the resources supported by AWS Config’s required tags functionality. Although that’s not the full list of resources, other resources are tagged through the custom code defined in the Automation Runbook.

Once the Cloud formation template is ready, upload it to the S3 bucket that was previously identified.

Deploy Conformance Pack as an Organization Conformance Pack

The template for Org conformance pack is as follows –

AWSTemplateFormatVersion: 2010-09-09
Description: This CloudFormation template creates a Organization conformance pack by using the conformance pack template defined in TemplateS3Uri
Parameters:
  ExcludedAccounts:
    Description: Comma delimited accounts in your organizations where you want to exclude from deploying this comformance pack
    Type: CommaDelimitedList
    Default: ''
  TemplateS3Url: 
    Type: String
Resources:
  TagRemediationConformancePack:
    Type: AWS::Config::OrganizationConformancePack
    Properties:
      ExcludedAccounts: !If [ ExcludedAccountsCheck, !Ref 'AWS::NoValue', !Ref ExcludedAccounts ]
      OrganizationConformancePackName: TagRemediationConformancePack
      TemplateS3Uri: !Ref TemplateS3Url
Conditions:
  ExcludedAccountsCheck:
    !Equals [!Select [0, !Ref ExcludedAccounts], '']

• While deploying this template, we will need a comma-separated list of AWS accounts that should be excluded from the deployment &
• The S3 path of the Conformance pack template was collected from the previous step. Please note that the accounts starting with Zero should include the zeros; otherwise, Cloud Formation will exit with an error.
• Once the stack is created, everything is in place. Based on this framework, AWS Config will remediate every detection and run the automation runbook for tagging.

Refer to our GitHub link for detailed code.

Conclusion

What we went through in this blog was to see how you can reactively fix your tagging issues and ensure that resources are compliant continuously. The ideal way of fixing tagging is to have an appropriate strategy that restricts the creation of new resources and proactively reports tag non-compliance. We are Versent and can always help you build an optimised strategy. We can also help you remediate your resources using the compliance framework with all the customisations included for every resource type.