cloud security

Performing Cloud-Native Threat Hunting with AWS Detective

Hunt for threats in AWS environments using Detective behavior graphs, entity investigation timelines, GuardDuty finding correlation, and automated entity profiling across IAM users, EC2 instances, and IP addresses.

awsaws-detectivebehavior-graphcloud-securityec2guarddutyiamincident-investigation
Install this skill
npx skills add mukul975/Anthropic-Cybersecurity-Skills
Framework mappings

Overview

AWS Detective automatically collects and analyzes log data from AWS CloudTrail, VPC Flow Logs, GuardDuty findings, and EKS audit logs to build interactive behavior graphs. These graphs enable security analysts to investigate entities (IAM users, roles, IP addresses, EC2 instances) across time, identify anomalous API calls, detect lateral movement between accounts, and correlate GuardDuty findings into coherent attack narratives — all without manual log parsing.

Prerequisites

  • AWS account with Detective enabled (requires GuardDuty active for 48+ hours)
  • AWS CLI v2 configured with appropriate IAM permissions (detective:*, guardduty:List*)
  • Python 3.9+ with boto3
  • IAM policy: AmazonDetectiveFullAccess or custom policy with detective:SearchGraph, detective:GetInvestigation, detective:ListIndicators

Key Concepts

Concept Description
Behavior Graph Data structure linking CloudTrail, VPC Flow, GuardDuty, and EKS logs for an account/region
Entity Investigable object: IAM user, IAM role, EC2 instance, IP address, S3 bucket, EKS cluster
Finding Group Correlated set of GuardDuty findings linked to the same attack campaign
Entity Profile Timeline of API calls, network connections, and resource access for a specific entity
Scope Time Investigation window (default 24h, max 1 year) for behavioral analysis

Steps

Step 1: List Available Behavior Graphs

aws detective list-graphs --output table

Step 2: Investigate a Suspicious IAM User

# Get entity profile for an IAM user
aws detective get-investigation \
  --graph-arn arn:aws:detective:us-east-1:123456789012:graph:a1b2c3d4 \
  --investigation-id 000000000000000000001

Step 3: Search Entities Programmatically

#!/usr/bin/env python3
"""Search AWS Detective for suspicious entities."""
import boto3
import json
from datetime import datetime, timedelta
 
detective = boto3.client('detective')
 
def list_behavior_graphs():
    """List all Detective behavior graphs."""
    response = detective.list_graphs()
    return response.get('GraphList', [])
 
def get_investigation_indicators(graph_arn, investigation_id, max_results=50):
    """Get indicators for a specific investigation."""
    response = detective.list_indicators(
        GraphArn=graph_arn,
        InvestigationId=investigation_id,
        MaxResults=max_results
    )
    return response.get('Indicators', [])
 
def investigate_guardduty_findings(graph_arn):
    """List high-severity investigations correlated by Detective."""
    response = detective.list_investigations(
        GraphArn=graph_arn,
        FilterCriteria={
            'Severity': {'Value': 'CRITICAL'},
            'Status': {'Value': 'RUNNING'}
        },
        MaxResults=20
    )
 
    for investigation in response.get('InvestigationDetails', []):
        print(f"Investigation: {investigation['InvestigationId']}")
        print(f"  Entity: {investigation['EntityArn']}")
        print(f"  Status: {investigation['Status']}")
        print(f"  Severity: {investigation['Severity']}")
        print(f"  Created: {investigation['CreatedTime']}")
        print()
 
if __name__ == "__main__":
    graphs = list_behavior_graphs()
    for graph in graphs:
        print(f"Graph: {graph['Arn']}")
        investigate_guardduty_findings(graph['Arn'])

Step 4: Analyze Finding Groups for Attack Campaigns

# List investigations with high severity
aws detective list-investigations \
  --graph-arn arn:aws:detective:us-east-1:123456789012:graph:a1b2c3d4 \
  --filter-criteria '{"Severity":{"Value":"HIGH"}}' \
  --max-results 10

Step 5: Check Entity Indicators

# Get indicators for a specific investigation
aws detective list-indicators \
  --graph-arn arn:aws:detective:us-east-1:123456789012:graph:a1b2c3d4 \
  --investigation-id 000000000000000000001 \
  --max-results 50

Expected Output

The list-investigations command returns investigation metadata:

{
  "InvestigationDetails": [
    {
      "InvestigationId": "000000000000000000001",
      "Severity": "CRITICAL",
      "Status": "RUNNING",
      "State": "ACTIVE",
      "EntityArn": "arn:aws:iam::123456789012:user/suspicious-user",
      "EntityType": "IAM_USER",
      "CreatedTime": "2026-03-15T14:30:00Z"
    }
  ]
}

Indicators are retrieved separately via list-indicators and include types such as TTP_OBSERVED, IMPOSSIBLE_TRAVEL, FLAGGED_IP_ADDRESS, NEW_GEOLOCATION, NEW_ASO, NEW_USER_AGENT, RELATED_FINDING, and RELATED_FINDING_GROUP.

Verification

  1. Confirm behavior graph has data: aws detective list-graphs returns non-empty list
  2. Validate investigation results contain entity timelines with API call sequences
  3. Cross-reference Detective findings with raw CloudTrail logs for accuracy
  4. Verify finding group correlations match manual investigation conclusions
  5. Confirm automated alerts trigger for HIGH/CRITICAL severity investigations
Source materials

References and resources

Everything below is rendered for inspection. Script files are read-only and never run.

References 3

api-reference.md8.5 KB

AWS Detective API Reference

This reference covers the Amazon Detective API for cloud-native threat hunting, via the AWS SDK for Python (boto3) and the AWS CLI. Detective ingests CloudTrail, VPC Flow Logs, GuardDuty findings, and EKS audit logs into a behavior graph and exposes entity profiles, finding groups, and guided investigations.

Authentication

Detective uses standard AWS IAM authentication — no separate API key. Credentials resolve through the SDK credential provider chain (environment variables, ~/.aws/credentials profile, EC2/ECS/EKS/Lambda role, or SSO).

import boto3
 
detective = boto3.client("detective", region_name="us-east-1")

Required IAM permissions (managed policy AmazonDetectiveFullAccess, or least-privilege custom):

Action Purpose
detective:ListGraphs Discover behavior graphs
detective:ListInvestigations List guided investigations
detective:GetInvestigation Get an investigation's results
detective:ListIndicators List indicators for an investigation
detective:StartInvestigation Launch a new investigation on an entity
detective:ListMembers / detective:GetMembers Multi-account graph membership
guardduty:ListFindings, guardduty:GetFindings Correlate GuardDuty findings

Prerequisite: Amazon GuardDuty must be enabled and active for at least 48 hours before Detective can build a usable behavior graph.

Key Methods (boto3 detective client)

Method Description Key Parameters
list_graphs List behavior graphs the account administers. MaxResults, NextToken
start_investigation Run an automated investigation on an entity over a scope window. GraphArn (required), EntityArn (required), ScopeStartTime, ScopeEndTime
get_investigation Retrieve an investigation's results (severity, status, scope, entity). GraphArn (required), InvestigationId (required)
list_investigations List investigations, filterable/sortable. GraphArn (required), FilterCriteria, SortCriteria, MaxResults, NextToken
list_indicators List indicators (TTPs, anomalies) tied to an investigation. GraphArn (required), InvestigationId (required), IndicatorType, MaxResults, NextToken
list_members / get_members Member accounts in the behavior graph. GraphArn, AccountIds
create_members / delete_members Invite/remove member accounts. GraphArn, Accounts
list_datasource_packages Optional data sources enabled (EKS audit, etc.). GraphArn
update_investigation_state Mark an investigation ARCHIVED / ACTIVE. GraphArn, InvestigationId, State

list_indicators — verified parameters

GraphArn (string, required), InvestigationId (string, required), IndicatorType (string, optional filter), NextToken (string — pagination token; expires after 24 hours), MaxResults (integer). Valid IndicatorType values:

TTP_OBSERVED · IMPOSSIBLE_TRAVEL · FLAGGED_IP_ADDRESS · NEW_GEOLOCATION · NEW_ASO (new autonomous system org) · NEW_USER_AGENT · RELATED_FINDING · RELATED_FINDING_GROUP

get_investigation — verified

Request: GraphArn (the behavior graph ARN), InvestigationId. Response includes CreatedTime (UTC ISO8601, e.g. 2021-08-18T16:35:56.284Z), EntityArn, EntityType, GraphArn, InvestigationId, ScopeStartTime, ScopeEndTime, plus severity/status/state.

list_investigations filter / sort detail

FilterCriteria = {
    "Severity":     {"Value": "CRITICAL"},   # INFORMATIONAL|LOW|MEDIUM|HIGH|CRITICAL
    "Status":       {"Value": "RUNNING"},     # RUNNING|FAILED|SUCCESSFUL
    "State":        {"Value": "ACTIVE"},      # ACTIVE|ARCHIVED
    "EntityArn":    {"Value": "arn:aws:iam::123456789012:user/suspicious"},
    "CreatedTime":  {"StartInclusive": <datetime>, "EndInclusive": <datetime>},
}
SortCriteria = {"Field": "SEVERITY", "SortOrder": "DESC"}  # CREATED_TIME|SEVERITY|STATUS

Python SDK

# Installation
pip install boto3
 
import boto3
 
detective = boto3.client("detective", region_name="us-east-1")
 
def hunt_critical(graph_arn):
    """List critical, currently-running investigations and their indicators."""
    inv = detective.list_investigations(
        GraphArn=graph_arn,
        FilterCriteria={
            "Severity": {"Value": "CRITICAL"},
            "Status":   {"Value": "RUNNING"},
        },
        SortCriteria={"Field": "SEVERITY", "SortOrder": "DESC"},
        MaxResults=20,
    )
    for d in inv.get("InvestigationDetails", []):
        print(d["InvestigationId"], d["EntityArn"], d["Severity"])
        ind = detective.list_indicators(
            GraphArn=graph_arn,
            InvestigationId=d["InvestigationId"],
            MaxResults=50,
        )
        for i in ind.get("Indicators", []):
            print("  ", i["IndicatorType"], i.get("IndicatorDetail"))
 
# Launch a fresh investigation on a suspect IAM principal
def investigate_entity(graph_arn, entity_arn, start, end):
    resp = detective.start_investigation(
        GraphArn=graph_arn,
        EntityArn=entity_arn,
        ScopeStartTime=start,   # datetime
        ScopeEndTime=end,       # datetime
    )
    return resp["InvestigationId"]
 
for g in detective.list_graphs().get("GraphList", []):
    hunt_critical(g["Arn"])

CLI equivalents:

aws detective list-graphs --output table
 
aws detective list-investigations \
  --graph-arn arn:aws:detective:us-east-1:123456789012:graph:abc \
  --filter-criteria '{"Severity":{"Value":"HIGH"}}' \
  --max-results 10
 
aws detective list-indicators \
  --graph-arn arn:aws:detective:us-east-1:123456789012:graph:abc \
  --investigation-id 000000000000000000001 --max-results 50

Common Response Fields

list_investigationsInvestigationDetails[]:

Field Meaning
InvestigationId Unique investigation ID
Severity INFORMATIONAL | LOW | MEDIUM | HIGH | CRITICAL
Status RUNNING | FAILED | SUCCESSFUL
State ACTIVE | ARCHIVED
EntityArn The entity under investigation
EntityType IAM_USER | IAM_ROLE (etc.)
CreatedTime Investigation creation timestamp (UTC ISO8601)

list_indicatorsIndicators[]: each has IndicatorType plus an IndicatorDetail union populated for the matching type (e.g. FlaggedIpAddressDetail, ImpossibleTravelDetail, NewGeolocationDetail, TTPsObservedDetail carrying MITRE ATT&CK tactic/technique).

Rate Limits / Service Quotas

Detective enforces account-level, per-Region quotas (most adjustable via Service Quotas):

Quota Default
Member accounts per behavior graph 1,200
Behavior graphs (administrator) per Region 1
Data retention in behavior graph 1 year of rolling history
Investigation scope window up to 1 year
Pagination token (list_indicators NextToken) lifetime 24 hours
API request rate Throttled per standard AWS API limits

Throttling returns TooManyRequestsException; boto3 retries with exponential backoff. There is no per-request monetary charge for the API itself — Detective is billed by volume of log data ingested into the behavior graph (GB/month, tiered).

Error Codes

Error Meaning
AccessDeniedException Caller lacks the required detective:* permission
ValidationException Invalid parameter (bad ARN, malformed filter)
ResourceNotFoundException Graph, investigation, or entity not found
TooManyRequestsException API rate quota exceeded; back off and retry
ConflictException Concurrent modification of graph membership
InternalServerException Transient service-side error; retry
ServiceQuotaExceededException Member/graph quota exceeded

Resources

standards.md1.1 KB

Standards & References

MITRE ATT&CK Cloud Matrix

  • TA0001 Initial Access: T1078 (Valid Accounts), T1190 (Exploit Public-Facing Application)
  • TA0003 Persistence: T1098 (Account Manipulation), T1136 (Create Account)
  • TA0004 Privilege Escalation: T1078, T1484 (Domain Policy Modification)
  • TA0005 Defense Evasion: T1562 (Impair Defenses), T1070 (Indicator Removal)
  • TA0006 Credential Access: T1528 (Steal Application Access Token)
  • TA0007 Discovery: T1580 (Cloud Infrastructure Discovery), T1526 (Cloud Service Discovery)
  • TA0009 Collection: T1530 (Data from Cloud Storage)
  • TA0010 Exfiltration: T1537 (Transfer Data to Cloud Account)

AWS Documentation

CIS AWS Foundations Benchmark

  • Section 4: Monitoring (relevant to Detective integration)
workflows.md1.1 KB

AWS Detective Investigation Workflow

Phase 1: Triage

  1. Review GuardDuty HIGH/CRITICAL findings
  2. Open Detective console → Finding Groups
  3. Identify clustered findings pointing to same entity

Phase 2: Entity Investigation

  1. Select entity (IAM user/role, EC2, IP)
  2. Review 24h behavior timeline
  3. Identify unusual API calls, new geolocations, impossible travel
  4. Check for privilege escalation patterns (CreateAccessKey, AttachPolicy)

Phase 3: Scope Assessment

  1. Trace lateral movement via AssumeRole chains
  2. Check S3 data access patterns
  3. Review VPC Flow Logs for unusual outbound connections
  4. Identify all compromised credentials

Phase 4: Correlation

  1. Map findings to MITRE ATT&CK techniques
  2. Build attack timeline from entity profiles
  3. Identify initial access vector
  4. Document indicators of compromise (IOCs)

Phase 5: Response

  1. Preserve evidence (CloudTrail logs, flow logs, snapshots) when safe
  2. Disable compromised credentials
  3. Revoke active sessions
  4. Isolate affected resources
  5. If active impact is ongoing, contain first and document evidence trade-offs

Scripts 1

process.py6.9 KB
Display-only source. This catalog never executes bundled scripts.
#!/usr/bin/env python3
"""
AWS Detective Threat Hunting Script

Lists behavior graphs, retrieves investigations, and analyzes entity
indicators for cloud-native threat hunting.
"""

import boto3
import json
import sys
import os
from datetime import datetime, timedelta


def _collect_all_pages(client_method, result_key, **kwargs):
    """Paginate through all pages of an AWS Detective API call."""
    all_items = []
    while True:
        response = client_method(**kwargs)
        all_items.extend(response.get(result_key, []))
        next_token = response.get('NextToken')
        if not next_token:
            break
        kwargs['NextToken'] = next_token
    return all_items


def list_behavior_graphs(session):
    """List all Detective behavior graphs in the account."""
    client = session.client('detective')
    graphs = _collect_all_pages(client.list_graphs, 'GraphList')

    if not graphs:
        print("[!] No behavior graphs found. Enable Detective first.")
        return []

    print(f"[+] Found {len(graphs)} behavior graph(s)\n")
    for graph in graphs:
        print(f"  ARN: {graph['Arn']}")
        created = graph.get('CreatedTime', 'N/A')
        print(f"  Created: {created}")
        print()

    return graphs


def list_investigations(session, graph_arn, severity=None, max_results=20):
    """List investigations filtered by severity."""
    client = session.client('detective')

    filter_criteria = {}
    if severity:
        filter_criteria['Severity'] = {'Value': severity}

    kwargs = {
        'GraphArn': graph_arn,
        'MaxResults': max_results,
    }
    if filter_criteria:
        kwargs['FilterCriteria'] = filter_criteria

    investigations = _collect_all_pages(
        client.list_investigations, 'InvestigationDetails', **kwargs
    )

    if not investigations:
        print("[+] No investigations found matching criteria")
        return []

    print(f"[+] Found {len(investigations)} investigation(s)\n")
    for inv in investigations:
        inv_id = inv.get('InvestigationId', 'N/A')
        severity = inv.get('Severity', 'N/A')
        status = inv.get('Status', 'N/A')
        entity = inv.get('EntityArn', 'N/A')
        created = inv.get('CreatedTime', 'N/A')
        print(f"  Investigation: {inv_id}")
        print(f"    Severity: {severity} | Status: {status}")
        print(f"    Entity: {entity}")
        print(f"    Created: {created}")
        print()

    return investigations


def get_investigation_detail(session, graph_arn, investigation_id):
    """Get detailed information about a specific investigation."""
    client = session.client('detective')

    response = client.get_investigation(
        GraphArn=graph_arn,
        InvestigationId=investigation_id,
    )

    print(f"[+] Investigation: {investigation_id}")
    print(f"  Entity: {response.get('EntityArn', 'N/A')}")
    print(f"  Entity Type: {response.get('EntityType', 'N/A')}")
    print(f"  Severity: {response.get('Severity', 'N/A')}")
    print(f"  Status: {response.get('Status', 'N/A')}")
    print(f"  Created: {response.get('CreatedTime', 'N/A')}")
    print(f"  Scope Start: {response.get('ScopeStartTime', 'N/A')}")
    print(f"  Scope End: {response.get('ScopeEndTime', 'N/A')}")

    return response


def list_indicators(session, graph_arn, investigation_id, max_results=50):
    """List indicators for a specific investigation."""
    client = session.client('detective')

    indicators = _collect_all_pages(
        client.list_indicators, 'Indicators',
        GraphArn=graph_arn,
        InvestigationId=investigation_id,
        MaxResults=max_results,
    )
    if not indicators:
        print("[+] No indicators found for this investigation")
        return []

    print(f"[+] Found {len(indicators)} indicator(s)\n")
    for ind in indicators:
        ind_type = ind.get('IndicatorType', 'N/A')
        detail = ind.get('IndicatorDetail', {})
        print(f"  Type: {ind_type}")
        if detail:
            print(f"    Detail: {json.dumps(detail, default=str)[:200]}")
        print()

    return indicators


def export_results(data, output_dir):
    """Export investigation results to JSON."""
    os.makedirs(output_dir, exist_ok=True)
    out_path = os.path.join(output_dir, "detective_results.json")
    with open(out_path, "w") as f:
        json.dump(data, f, indent=2, default=str)
    print(f"[+] Results exported to {out_path}")
    return out_path


if __name__ == "__main__":
    import argparse

    parser = argparse.ArgumentParser(
        description="AWS Detective Threat Hunting Tool"
    )
    parser.add_argument(
        "--graphs", action="store_true", help="List behavior graphs"
    )
    parser.add_argument(
        "--investigations", action="store_true", help="List investigations"
    )
    parser.add_argument("--graph-arn", type=str, help="Behavior graph ARN")
    parser.add_argument(
        "--investigation-id", type=str, help="Investigation ID for detail view"
    )
    parser.add_argument(
        "--indicators", action="store_true", help="List indicators"
    )
    parser.add_argument(
        "--severity",
        type=str,
        default=None,
        choices=["INFORMATIONAL", "LOW", "MEDIUM", "HIGH", "CRITICAL"],
        help="Severity filter (e.g. HIGH)",
    )
    parser.add_argument("--max-results", type=int, default=20,
                        help="Max results per API call (1-100)")
    parser.add_argument("--region", default="us-east-1")
    parser.add_argument("--profile", type=str, help="AWS profile name")
    parser.add_argument(
        "--output", type=str, help="Output directory for JSON export"
    )
    args = parser.parse_args()

    if args.max_results < 1 or args.max_results > 100:
        parser.error("--max-results must be between 1 and 100")

    kwargs = {"region_name": args.region}
    if args.profile:
        kwargs["profile_name"] = args.profile
    session = boto3.Session(**kwargs)

    results = {}

    if args.graphs:
        results["graphs"] = list_behavior_graphs(session)

    if args.investigations:
        if not args.graph_arn:
            print("[!] --graph-arn required for --investigations")
            sys.exit(1)
        results["investigations"] = list_investigations(
            session, args.graph_arn, args.severity, args.max_results
        )

    if args.investigation_id:
        if not args.graph_arn:
            print("[!] --graph-arn required for --investigation-id")
            sys.exit(1)
        results["detail"] = get_investigation_detail(
            session, args.graph_arn, args.investigation_id
        )

    if args.indicators:
        if not args.graph_arn or not args.investigation_id:
            print("[!] --graph-arn and --investigation-id required for --indicators")
            sys.exit(1)
        results["indicators"] = list_indicators(
            session, args.graph_arn, args.investigation_id, args.max_results
        )

    if args.output and results:
        export_results(results, args.output)

Assets 1

template.mdtext/markdown · 1.1 KB
Keep exploring