cloud security

Remediating S3 Bucket Misconfiguration

This skill provides step-by-step procedures for identifying and remediating Amazon S3 bucket misconfigurations that expose sensitive data to unauthorized access. It covers enabling S3 Block Public Access at account and bucket levels, auditing bucket policies and ACLs, enforcing encryption, configuring access logging, and deploying automated remediation using AWS Config and Lambda.

aws-configbucket-misconfigurationdata-exposurepublic-access-blocks3-security
Install this skill
npx skills add mukul975/Anthropic-Cybersecurity-Skills
Framework mappings

When to Use

  • When AWS Config or Security Hub reports S3 buckets with public access or missing encryption
  • When a security scan reveals S3 bucket policies granting access to Principal "*" (everyone)
  • When preparing for a data protection audit requiring evidence of storage security controls
  • When responding to a data exposure incident involving publicly accessible S3 objects
  • When establishing preventive controls for new S3 bucket creation across an AWS Organization

Do not use for Azure Blob Storage or GCP Cloud Storage misconfigurations, for S3 data classification (see implementing-cloud-dlp-policy), or for S3 access pattern analysis unrelated to security.

Prerequisites

  • AWS account with S3 administrative permissions (s3:, s3-outposts:)
  • AWS Config enabled to evaluate S3 resource compliance
  • AWS CloudTrail logging S3 data events for access auditing
  • Macie enabled for sensitive data discovery in S3 buckets

Workflow

Step 1: Identify All Public and Misconfigured Buckets

Use multiple detection methods to identify S3 buckets with public access. Rely on AWS Config rules, S3 Access Analyzer, and Macie rather than manual inspection.

# Enable S3 Access Analyzer for external access detection
aws accessanalyzer create-analyzer \
  --analyzer-name s3-analyzer \
  --type ACCOUNT
 
# List all S3 buckets with public access indicators
aws s3api list-buckets --query 'Buckets[*].Name' --output text | while read bucket; do
  public_status=$(aws s3api get-public-access-block --bucket "$bucket" 2>/dev/null)
  if [ $? -ne 0 ]; then
    echo "NO PUBLIC ACCESS BLOCK: $bucket"
  fi
done
 
# Check bucket policies for public access grants
aws s3api list-buckets --query 'Buckets[*].Name' --output text | while read bucket; do
  policy=$(aws s3api get-bucket-policy --bucket "$bucket" 2>/dev/null)
  if echo "$policy" | grep -q '"Principal":"*"' 2>/dev/null; then
    echo "PUBLIC POLICY DETECTED: $bucket"
  fi
done
 
# Use AWS Config to find non-compliant buckets
aws configservice get-compliance-details-by-config-rule \
  --config-rule-name s3-bucket-public-read-prohibited \
  --compliance-types NON_COMPLIANT \
  --query 'EvaluationResults[*].EvaluationResultIdentifier.EvaluationResultQualifier.ResourceId'

Step 2: Enable S3 Block Public Access at Account Level

Apply the four Block Public Access settings at the AWS account level as a safety net. This prevents any bucket in the account from being made public, regardless of individual bucket policies or ACLs.

# Enable account-level Block Public Access (all four settings)
aws s3control put-public-access-block \
  --account-id 123456789012 \
  --public-access-block-configuration '{
    "BlockPublicAcls": true,
    "IgnorePublicAcls": true,
    "BlockPublicPolicy": true,
    "RestrictPublicBuckets": true
  }'
 
# Verify account-level settings
aws s3control get-public-access-block --account-id 123456789012
 
# Enable at bucket level for defense in depth
aws s3api put-public-access-block \
  --bucket production-data-bucket \
  --public-access-block-configuration '{
    "BlockPublicAcls": true,
    "IgnorePublicAcls": true,
    "BlockPublicPolicy": true,
    "RestrictPublicBuckets": true
  }'

Step 3: Audit and Remediate Bucket Policies and ACLs

Review all bucket policies for overly permissive Principal statements and remove legacy ACLs. Enforce bucket ownership controls to disable ACLs entirely.

# Remove a public bucket policy
aws s3api delete-bucket-policy --bucket exposed-bucket
 
# Replace with a restrictive policy
aws s3api put-bucket-policy --bucket exposed-bucket --policy '{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyUnencryptedTransport",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::exposed-bucket",
        "arn:aws:s3:::exposed-bucket/*"
      ],
      "Condition": {
        "Bool": {"aws:SecureTransport": "false"}
      }
    },
    {
      "Sid": "AllowOnlyVPCEndpoint",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::exposed-bucket",
        "arn:aws:s3:::exposed-bucket/*"
      ],
      "Condition": {
        "StringNotEquals": {"aws:SourceVpce": "vpce-0abc123def456"}
      }
    }
  ]
}'
 
# Enforce bucket owner for all objects (disable ACLs)
aws s3api put-bucket-ownership-controls --bucket exposed-bucket \
  --ownership-controls '{"Rules": [{"ObjectOwnership": "BucketOwnerEnforced"}]}'

Step 4: Enforce Default Encryption

Enable default server-side encryption with AWS KMS or AES-256 for all buckets. Add a bucket policy denying unencrypted object uploads.

# Enable default KMS encryption
aws s3api put-bucket-encryption --bucket production-data-bucket \
  --server-side-encryption-configuration '{
    "Rules": [{
      "ApplyServerSideEncryptionByDefault": {
        "SSEAlgorithm": "aws:kms",
        "KMSMasterKeyID": "arn:aws:kms:us-east-1:123456789012:key/key-id"
      },
      "BucketKeyEnabled": true
    }]
  }'
 
# Deny unencrypted uploads via bucket policy
aws s3api put-bucket-policy --bucket production-data-bucket --policy '{
  "Version": "2012-10-17",
  "Statement": [{
    "Sid": "DenyUnencryptedUploads",
    "Effect": "Deny",
    "Principal": "*",
    "Action": "s3:PutObject",
    "Resource": "arn:aws:s3:::production-data-bucket/*",
    "Condition": {
      "StringNotEquals": {"s3:x-amz-server-side-encryption": ["aws:kms", "AES256"]}
    }
  }]
}'

Step 5: Enable Access Logging and Monitoring

Configure S3 server access logging and CloudTrail data events to track all object-level operations. Set up EventBridge rules to alert on suspicious access patterns.

# Enable server access logging
aws s3api put-bucket-logging --bucket production-data-bucket \
  --bucket-logging-status '{
    "LoggingEnabled": {
      "TargetBucket": "s3-access-logs-bucket",
      "TargetPrefix": "production-data-bucket/"
    }
  }'
 
# Enable CloudTrail S3 data events
aws cloudtrail put-event-selectors --trail-name management-trail \
  --event-selectors '[{
    "ReadWriteType": "All",
    "DataResources": [{
      "Type": "AWS::S3::Object",
      "Values": ["arn:aws:s3:::production-data-bucket/"]
    }]
  }]'

Step 6: Deploy Preventive Controls with SCP and Config

Use Service Control Policies to prevent disabling Block Public Access across the organization. Deploy AWS Config rules with auto-remediation.

# SCP preventing Block Public Access removal
aws organizations create-policy \
  --name PreventS3PublicAccess \
  --type SERVICE_CONTROL_POLICY \
  --content '{
    "Version": "2012-10-17",
    "Statement": [{
      "Sid": "DenyRemovePublicAccessBlock",
      "Effect": "Deny",
      "Action": [
        "s3:PutBucketPublicAccessBlock",
        "s3:PutAccountPublicAccessBlock"
      ],
      "Resource": "*",
      "Condition": {
        "StringNotLike": {"aws:PrincipalArn": "arn:aws:iam::*:role/SecurityAdmin"}
      }
    }]
  }'

Key Concepts

Term Definition
S3 Block Public Access Four account-level and bucket-level settings that override any policy or ACL granting public access to S3 resources
Bucket Policy JSON-based resource policy attached to an S3 bucket defining who can access what objects under which conditions
ACL (Access Control List) Legacy S3 access mechanism that grants permissions at the bucket or object level; should be disabled via BucketOwnerEnforced
BucketOwnerEnforced Ownership control setting that disables all ACLs on a bucket, making the bucket owner the sole authority for access control
Server-Side Encryption Automatic encryption of objects at rest using AES-256 (SSE-S3), AWS KMS (SSE-KMS), or customer-provided keys (SSE-C)
VPC Endpoint Private connection between a VPC and S3 that restricts bucket access to traffic originating from within the VPC
S3 Access Analyzer IAM Access Analyzer capability that identifies S3 buckets shared with external entities outside the account or organization

Tools & Systems

  • AWS Config: Evaluates S3 bucket compliance against managed rules and triggers auto-remediation for non-compliant resources
  • Amazon Macie: Discovers and classifies sensitive data in S3 buckets to identify which misconfigurations pose the highest data exposure risk
  • IAM Access Analyzer: Identifies S3 buckets with policies or ACLs that grant access to external principals
  • S3 Storage Lens: Provides organization-wide visibility into S3 usage patterns, access metrics, and security anomalies
  • Prowler: Open-source tool that checks S3 security configurations against CIS benchmarks and best practices

Common Scenarios

Scenario: Data Breach from Publicly Readable S3 Bucket Containing PII

Context: A security researcher reports that an S3 bucket containing 273,000 bank transfer PDFs is publicly readable. The bucket was created by a developer who needed to share files with an external partner and set the ACL to public-read.

Approach:

  1. Immediately enable Block Public Access on the specific bucket to stop the exposure
  2. Revoke all public ACLs by setting BucketOwnerEnforced ownership controls
  3. Audit CloudTrail and S3 access logs to determine which IP addresses accessed the exposed objects
  4. Run Macie on the bucket to classify the types of PII exposed and assess regulatory notification requirements
  5. Enable account-level Block Public Access to prevent recurrence across all buckets
  6. Deploy an SCP preventing any principal except SecurityAdmin from modifying Block Public Access settings
  7. Create a pre-signed URL mechanism or S3 Access Point for the legitimate partner sharing use case

Pitfalls: Enabling Block Public Access without notifying the team that set up the public access breaks their workflow. Not running access log analysis before remediation loses evidence of who accessed the exposed data.

Output Format

S3 Bucket Security Remediation Report
=======================================
Account: 123456789012
Assessment Date: 2025-02-23
Buckets Scanned: 156
 
ACCOUNT-LEVEL CONTROLS:
  Block Public Access: ENABLED (all four settings)
  SCP Preventing Removal: DEPLOYED
 
CRITICAL FINDINGS (Remediated):
  [S3-001] production-uploads - Public READ via ACL
    Status: REMEDIATED - BucketOwnerEnforced applied
    Objects Exposed: 273,412
    Duration of Exposure: 47 days
    Unique External IPs Accessed: 1,247
 
  [S3-002] analytics-export - Public bucket policy (Principal: *)
    Status: REMEDIATED - Policy replaced with VPC endpoint restriction
    Sensitive Data (Macie): 12,400 objects with PII detected
 
HIGH FINDINGS:
  [S3-003] 14 buckets missing default encryption
    Status: REMEDIATED - KMS encryption enabled
  [S3-004] 8 buckets without server access logging
    Status: REMEDIATED - Logging enabled to centralized log bucket
 
SUMMARY:
  Buckets Remediated: 24/156
  Encryption Coverage: 100%
  Access Logging Coverage: 100%
  Block Public Access: 156/156 buckets
Source materials

References and resources

Everything below is rendered for inspection. Script files are read-only and never run.

References 1

api-reference.md2.5 KB

API Reference: S3 Bucket Misconfiguration Remediation Agent

Overview

Audits and remediates S3 bucket security: public access blocks, bucket policies, ACLs, encryption, versioning, and access logging using boto3.

Dependencies

Package Version Purpose
boto3 >= 1.28 AWS S3 API for audit and remediation

Audit Functions

check_public_access_block(s3, bucket)

Verifies all four S3 Block Public Access settings are enabled.

  • Returns: dict with block_config, fully_blocked

check_bucket_policy(s3, bucket)

Parses bucket policy for Principal: "*" Allow statements.

  • Returns: dict with public_statements list (risk: CRITICAL)

check_bucket_acl(s3, bucket)

Checks ACL grants for AllUsers or AuthenticatedUsers URIs.

  • Returns: dict with public_grants list

check_encryption(s3, bucket)

Checks for default server-side encryption configuration.

  • Returns: dict with encrypted, algorithm (AES256 or aws:kms)

check_versioning(s3, bucket)

Checks versioning status and MFA Delete configuration.

  • Returns: dict with status, mfa_delete

check_logging(s3, bucket)

Verifies access logging is enabled with target bucket.

  • Returns: dict with logging_enabled, target_bucket

audit_all_buckets(s3)

Full audit across all buckets, sorted by issue count.

  • Returns: list[dict] with risk rating per bucket

Remediation Functions

enable_public_access_block(s3, bucket)

Enables all four S3 Block Public Access settings.

enable_encryption(s3, bucket, algorithm)

Configures default SSE-KMS or AES256 encryption with bucket key.

enable_versioning(s3, bucket)

Enables S3 versioning on the bucket.

AWS API Calls

API Call Purpose
list_buckets Enumerate all buckets
get_public_access_block Check block config
put_public_access_block Apply block config
get_bucket_policy Read bucket policy
get_bucket_acl Read ACL grants
get_bucket_encryption Check encryption
put_bucket_encryption Enable encryption
get_bucket_versioning Check versioning
put_bucket_versioning Enable versioning
get_bucket_logging Check access logging

Environment Variables

Variable Required Description
AWS_ACCESS_KEY_ID Yes AWS credential
AWS_SECRET_ACCESS_KEY Yes AWS credential
AWS_DEFAULT_REGION No Default: us-east-1

Usage

python agent.py us-east-1

Scripts 1

agent.py7.5 KB
Display-only source. This catalog never executes bundled scripts.
#!/usr/bin/env python3
"""S3 bucket misconfiguration remediation agent using boto3."""

import json
import sys

try:
    import boto3
    from botocore.exceptions import ClientError
except ImportError:
    print("Install: pip install boto3")
    sys.exit(1)


def get_s3_client(region="us-east-1"):
    return boto3.client("s3", region_name=region)


def list_all_buckets(s3):
    """List all S3 buckets in the account."""
    resp = s3.list_buckets()
    return [b["Name"] for b in resp.get("Buckets", [])]


def check_public_access_block(s3, bucket):
    """Check if S3 Block Public Access is enabled."""
    try:
        config = s3.get_public_access_block(Bucket=bucket)
        block = config["PublicAccessBlockConfiguration"]
        all_blocked = all([
            block.get("BlockPublicAcls", False),
            block.get("IgnorePublicAcls", False),
            block.get("BlockPublicPolicy", False),
            block.get("RestrictPublicBuckets", False),
        ])
        return {"bucket": bucket, "block_config": block, "fully_blocked": all_blocked}
    except ClientError as e:
        if e.response["Error"]["Code"] == "NoSuchPublicAccessBlockConfiguration":
            return {"bucket": bucket, "block_config": None, "fully_blocked": False}
        raise


def check_bucket_policy(s3, bucket):
    """Check bucket policy for public access grants."""
    try:
        policy = json.loads(s3.get_bucket_policy(Bucket=bucket)["Policy"])
        findings = []
        for stmt in policy.get("Statement", []):
            principal = stmt.get("Principal", {})
            effect = stmt.get("Effect", "")
            if principal == "*" or principal == {"AWS": "*"}:
                if effect == "Allow":
                    findings.append({
                        "sid": stmt.get("Sid", "unnamed"),
                        "effect": effect,
                        "principal": str(principal),
                        "action": stmt.get("Action"),
                        "risk": "CRITICAL",
                    })
        return {"bucket": bucket, "has_policy": True, "public_statements": findings}
    except ClientError as e:
        if e.response["Error"]["Code"] == "NoSuchBucketPolicy":
            return {"bucket": bucket, "has_policy": False, "public_statements": []}
        raise


def check_bucket_acl(s3, bucket):
    """Check bucket ACL for public grants."""
    acl = s3.get_bucket_acl(Bucket=bucket)
    public_grants = []
    for grant in acl.get("Grants", []):
        grantee = grant.get("Grantee", {})
        uri = grantee.get("URI", "")
        if "AllUsers" in uri or "AuthenticatedUsers" in uri:
            public_grants.append({
                "grantee": uri,
                "permission": grant.get("Permission"),
                "risk": "CRITICAL" if grant.get("Permission") in ("FULL_CONTROL", "WRITE") else "HIGH",
            })
    return {"bucket": bucket, "public_grants": public_grants}


def check_encryption(s3, bucket):
    """Check if default encryption is enabled."""
    try:
        config = s3.get_bucket_encryption(Bucket=bucket)
        rules = config.get("ServerSideEncryptionConfiguration", {}).get("Rules", [])
        encryption = None
        for rule in rules:
            sse = rule.get("ApplyServerSideEncryptionByDefault", {})
            encryption = sse.get("SSEAlgorithm")
        return {"bucket": bucket, "encrypted": True, "algorithm": encryption}
    except ClientError as e:
        if e.response["Error"]["Code"] == "ServerSideEncryptionConfigurationNotFoundError":
            return {"bucket": bucket, "encrypted": False, "algorithm": None}
        raise


def check_versioning(s3, bucket):
    """Check if versioning is enabled."""
    resp = s3.get_bucket_versioning(Bucket=bucket)
    return {
        "bucket": bucket,
        "status": resp.get("Status", "Disabled"),
        "mfa_delete": resp.get("MFADelete", "Disabled"),
    }


def check_logging(s3, bucket):
    """Check if access logging is enabled."""
    resp = s3.get_bucket_logging(Bucket=bucket)
    logging_config = resp.get("LoggingEnabled")
    return {
        "bucket": bucket,
        "logging_enabled": logging_config is not None,
        "target_bucket": logging_config.get("TargetBucket") if logging_config else None,
    }


def enable_public_access_block(s3, bucket):
    """Enable S3 Block Public Access on a bucket."""
    s3.put_public_access_block(
        Bucket=bucket,
        PublicAccessBlockConfiguration={
            "BlockPublicAcls": True,
            "IgnorePublicAcls": True,
            "BlockPublicPolicy": True,
            "RestrictPublicBuckets": True,
        },
    )
    return {"bucket": bucket, "action": "block_public_access", "status": "applied"}


def enable_encryption(s3, bucket, algorithm="aws:kms"):
    """Enable default encryption on a bucket."""
    config = {
        "Rules": [{
            "ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": algorithm},
            "BucketKeyEnabled": True,
        }]
    }
    s3.put_bucket_encryption(
        Bucket=bucket,
        ServerSideEncryptionConfiguration=config,
    )
    return {"bucket": bucket, "action": "enable_encryption", "algorithm": algorithm}


def enable_versioning(s3, bucket):
    """Enable versioning on a bucket."""
    s3.put_bucket_versioning(
        Bucket=bucket,
        VersioningConfiguration={"Status": "Enabled"},
    )
    return {"bucket": bucket, "action": "enable_versioning", "status": "Enabled"}


def audit_all_buckets(s3):
    """Run full security audit across all buckets."""
    buckets = list_all_buckets(s3)
    results = []
    for bucket in buckets:
        finding = {"bucket": bucket, "issues": []}
        pab = check_public_access_block(s3, bucket)
        if not pab["fully_blocked"]:
            finding["issues"].append("Public access block not fully enabled")
        policy = check_bucket_policy(s3, bucket)
        if policy["public_statements"]:
            finding["issues"].append(f"{len(policy['public_statements'])} public policy statement(s)")
        acl = check_bucket_acl(s3, bucket)
        if acl["public_grants"]:
            finding["issues"].append(f"{len(acl['public_grants'])} public ACL grant(s)")
        enc = check_encryption(s3, bucket)
        if not enc["encrypted"]:
            finding["issues"].append("No default encryption")
        ver = check_versioning(s3, bucket)
        if ver["status"] != "Enabled":
            finding["issues"].append("Versioning disabled")
        log = check_logging(s3, bucket)
        if not log["logging_enabled"]:
            finding["issues"].append("Access logging disabled")
        finding["issue_count"] = len(finding["issues"])
        finding["risk"] = "CRITICAL" if any("public" in i.lower() for i in finding["issues"]) else (
            "HIGH" if finding["issue_count"] >= 3 else "MEDIUM" if finding["issue_count"] >= 1 else "LOW"
        )
        results.append(finding)
    return sorted(results, key=lambda x: -x["issue_count"])


def print_audit_report(results):
    print("S3 Bucket Security Audit Report")
    print("=" * 50)
    print(f"Buckets Audited: {len(results)}")
    critical = sum(1 for r in results if r["risk"] == "CRITICAL")
    print(f"Critical: {critical}")
    for r in results:
        if r["issue_count"] == 0:
            continue
        print(f"\n[{r['risk']}] {r['bucket']} ({r['issue_count']} issues)")
        for issue in r["issues"]:
            print(f"  - {issue}")


if __name__ == "__main__":
    region = sys.argv[1] if len(sys.argv) > 1 else "us-east-1"
    s3 = get_s3_client(region)
    results = audit_all_buckets(s3)
    print_audit_report(results)
Keep exploring