soc operations

Investigating Insider Threat Indicators

Investigates insider threat indicators including data exfiltration attempts, unauthorized access patterns, policy violations, and pre-departure behaviors using SIEM analytics, DLP alerts, and HR data correlation. Use when SOC teams receive insider threat referrals from HR, detect anomalous data movement by employees, or need to build investigation timelines for potential insider threats.

data-exfiltrationdlphr-correlationinsider-threatinvestigationsocueba
Install this skill
npx skills add mukul975/Anthropic-Cybersecurity-Skills
Framework mappings

When to Use

Use this skill when:

  • HR refers a departing employee for monitoring during their notice period
  • DLP alerts indicate bulk data downloads or transfers to personal storage
  • UEBA detects anomalous access patterns deviating significantly from peer baselines
  • Management reports concerns about an employee accessing sensitive data outside their role

Do not use without proper legal authorization — insider threat investigations must be coordinated with HR, Legal, and Privacy teams before monitoring begins.

Prerequisites

  • Legal authorization and HR referral documenting investigation justification
  • SIEM with DLP, endpoint, email, proxy, and authentication log sources
  • Data Loss Prevention (DLP) system (Microsoft Purview, Symantec, Forcepoint) with policy alerts
  • Endpoint monitoring capability (EDR with USB/removable media logging)
  • HR data feed providing employment status, notice dates, and access entitlements
  • Chain of custody procedures for evidence preservation

Workflow

Step 1: Establish Investigation Scope and Legal Authorization

Before any monitoring, ensure proper authorization:

INSIDER THREAT INVESTIGATION AUTHORIZATION
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Case ID:           IT-2024-0089
Subject:           [Employee Name] — [Department]
Authorized By:     [CISO / General Counsel]
Referral Source:   HR — Employee submitted resignation, 2-week notice
Justification:     Employee has access to trade secrets and customer PII
Scope:             Email, file access, USB, cloud storage, printing
Duration:          2024-03-15 to 2024-03-29 (notice period)
Privacy Review:    Completed — compliant with acceptable use policy

Step 2: Build Activity Timeline from SIEM

Query comprehensive activity for the subject:

index=* (user="jsmith" OR src_user="jsmith" OR sender="jsmith@company.com"
         OR SubjectUserName="jsmith")
earliest="2024-03-01" latest=now
| eval event_category = case(
    sourcetype LIKE "%dlp%", "DLP",
    sourcetype LIKE "%proxy%", "Web Access",
    sourcetype LIKE "%email%", "Email",
    sourcetype LIKE "%WinEventLog%", "Endpoint",
    sourcetype LIKE "%o365%", "Cloud",
    sourcetype LIKE "%vpn%", "VPN",
    sourcetype LIKE "%badge%", "Physical Access",
    1=1, sourcetype
  )
| stats count by event_category, sourcetype, _time
| timechart span=1d count by event_category

Step 3: Detect Data Exfiltration Indicators

Bulk File Downloads (SharePoint/OneDrive):

index=o365 sourcetype="o365:management:activity" Operation IN ("FileDownloaded", "FileSynced")
UserId="jsmith@company.com" earliest=-30d
| stats count AS downloads, sum(eval(if(isnotnull(FileSize), FileSize, 0))) AS total_bytes,
        dc(SourceFileName) AS unique_files
  by UserId, SiteUrl, _time
| bin _time span=1d
| eval total_gb = round(total_bytes / 1073741824, 2)
| where downloads > 50 OR total_gb > 1
| sort - total_gb

USB/Removable Media Usage:

index=sysmon EventCode=1 Computer="WORKSTATION-JSMITH"
(CommandLine="*removable*" OR CommandLine="*usb*"
 OR Image="*\\xcopy*" OR Image="*\\robocopy*")
| table _time, Computer, User, Image, CommandLine
| append [
    search index=endpoint sourcetype="endpoint:device_connect"
    user="jsmith" device_type="removable"
    | table _time, user, device_name, device_serial, action
  ]
| sort _time

Email-Based Exfiltration:

index=email sourcetype="o365:messageTrace"
SenderAddress="jsmith@company.com"
| eval is_external = if(match(RecipientAddress, "@company\.com$"), 0, 1)
| eval has_attachment = if(isnotnull(AttachmentName), 1, 0)
| stats count AS total_emails,
        sum(is_external) AS external_emails,
        sum(has_attachment) AS with_attachments,
        sum(eval(if(is_external=1 AND has_attachment=1, 1, 0))) AS external_with_attach,
        sum(Size) AS total_size_bytes
  by SenderAddress
| eval external_attach_pct = round(external_with_attach / total_emails * 100, 1)
| eval total_size_mb = round(total_size_bytes / 1048576, 1)

Cloud Storage Upload Detection:

index=proxy user="jsmith"
(dest IN ("*dropbox.com", "*drive.google.com", "*onedrive.live.com",
          "*box.com", "*wetransfer.com", "*mega.nz")
 OR category="cloud-storage")
http_method=POST
| stats count AS uploads, sum(bytes_out) AS total_uploaded
  by user, dest, category
| eval uploaded_mb = round(total_uploaded / 1048576, 1)
| sort - uploaded_mb

Step 4: Analyze Access Pattern Anomalies

Accessing Sensitive Systems Outside Normal Scope:

index=auth user="jsmith" action=success earliest=-30d
| stats dc(app) AS unique_apps, values(app) AS apps_accessed by user
| join user type=left [
    | inputlookup role_app_mapping.csv
    | search role="Financial Analyst"
    | stats values(authorized_app) AS authorized_apps by role
    | eval user="jsmith"
  ]
| eval unauthorized = mvfilter(NOT match(apps_accessed, mvjoin(authorized_apps, "|")))
| where isnotnull(unauthorized)
| table user, unauthorized, authorized_apps

After-Hours and Weekend Activity:

index=* user="jsmith" earliest=-30d
| eval hour = tonumber(strftime(_time, "%H"))
| eval is_offhours = if(hour < 7 OR hour > 19, 1, 0)
| eval day = strftime(_time, "%A")
| eval is_weekend = if(day IN ("Saturday", "Sunday"), 1, 0)
| stats count AS total, sum(is_offhours) AS offhours, sum(is_weekend) AS weekend by user
| eval offhours_pct = round(offhours / total * 100, 1)
| eval weekend_pct = round(weekend / total * 100, 1)

Step 5: Correlate with HR and Physical Security Data

Compare activity to resignation timeline:

| makeresults
| eval user="jsmith",
       resignation_date="2024-03-15",
       last_day="2024-03-29",
       access_revocation="2024-03-29 17:00"
| join user [
    search index=* user="jsmith" earliest=-90d
    | bin _time span=1d
    | stats count AS daily_events, dc(sourcetype) AS data_sources by user, _time
  ]
| eval phase = case(
    _time < relative_time(now(), "-30d"), "Normal (Pre-Resignation)",
    _time >= strptime(resignation_date, "%Y-%m-%d") AND _time <= strptime(last_day, "%Y-%m-%d"),
      "Notice Period",
    1=1, "Transition"
  )
| chart avg(daily_events) AS avg_events by phase

Badge/Physical Access Correlation:

index=badge_access employee_id="jsmith" earliest=-30d
| stats count AS badge_events, values(door_name) AS doors_accessed,
        earliest(_time) AS first_badge, latest(_time) AS last_badge by employee_id
| eval areas = mvcount(doors_accessed)

Step 6: Preserve Evidence and Document Findings

Maintain chain of custody for all collected evidence:

import hashlib
import json
from datetime import datetime
 
evidence_log = {
    "case_id": "IT-2024-0089",
    "investigator": "soc_analyst_tier2",
    "collection_time": datetime.utcnow().isoformat(),
    "items": [
        {
            "item_id": "EV-001",
            "description": "Splunk export — all user activity 2024-03-01 to 2024-03-15",
            "file": "jsmith_activity_export.csv",
            "sha256": hashlib.sha256(open("jsmith_activity_export.csv", "rb").read()).hexdigest(),
            "collected_by": "analyst_doe",
            "collection_method": "Splunk search export"
        },
        {
            "item_id": "EV-002",
            "description": "DLP alert details — 47 policy violations",
            "file": "dlp_alerts_jsmith.json",
            "sha256": hashlib.sha256(open("dlp_alerts_jsmith.json", "rb").read()).hexdigest(),
            "collected_by": "analyst_doe",
            "collection_method": "Microsoft Purview export"
        }
    ]
}
 
with open(f"evidence_log_{evidence_log['case_id']}.json", "w") as f:
    json.dump(evidence_log, f, indent=2)

Key Concepts

Term Definition
Insider Threat Risk posed by individuals with legitimate access who misuse it for unauthorized purposes
Data Exfiltration Unauthorized transfer of data outside the organization via email, USB, cloud, or other channels
DLP Data Loss Prevention — technology monitoring and blocking unauthorized data transfers based on content policies
Notice Period Monitoring Enhanced surveillance of departing employees during their resignation-to-departure window
Chain of Custody Documented evidence handling procedures ensuring forensic integrity for potential legal proceedings
Need-to-Know Violation Accessing information or systems beyond what is required for an employee's role or current tasks

Tools & Systems

  • Microsoft Purview (formerly DLP): Data classification and loss prevention platform monitoring endpoints, email, and cloud storage
  • Splunk UBA: User behavior analytics detecting insider threat patterns through ML-based anomaly detection
  • Forcepoint Insider Threat: Dedicated insider threat detection platform with behavioral indicators and risk scoring
  • DTEX InTERCEPT: Endpoint-based insider threat detection focusing on user activity metadata collection
  • Code42 Incydr: Data risk detection platform specializing in file exfiltration monitoring across endpoints and cloud

Common Scenarios

  • Departing Employee: Bulk download of customer lists and product roadmaps during two-week notice period
  • Disgruntled Employee: After negative performance review, employee accesses executive salary data outside their role
  • Contractor Overreach: External consultant accessing systems beyond contracted scope, downloading source code
  • Account Misuse: Employee sharing credentials with unauthorized third party for competitive intelligence
  • Sabotage Indicator: IT admin creating backdoor accounts and modifying system configurations before departure

Output Format

INSIDER THREAT INVESTIGATION REPORT — IT-2024-0089
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Subject:      jsmith (Financial Analyst, Finance Dept)
Period:       2024-03-01 to 2024-03-15
Status:       Employee resigned 2024-03-15, last day 2024-03-29
 
Key Findings:
  [HIGH]  3,847 files downloaded from SharePoint (12.4 GB) — 10x peer average
  [HIGH]  USB device connected 14 times during notice period (0 times prior month)
  [HIGH]  187 emails with attachments sent to personal Gmail
  [MEDIUM] After-hours activity increased 340% during notice period
  [MEDIUM] Accessed HR salary database 3 times (not authorized for role)
 
Timeline:
  Mar 01-14:  Normal activity baseline (avg 150 events/day)
  Mar 15:     Resignation submitted (activity spike to 890 events)
  Mar 16-17:  Weekend access — 2,100 SharePoint downloads
  Mar 18:     USB device first connected, DLP alert triggered
 
Evidence Collected:   4 items (SHA-256 verified, chain of custody documented)
Recommendation:       Immediate access revocation recommended
                      Evidence package prepared for Legal review
Source materials

References and resources

Everything below is rendered for inspection. Script files are read-only and never run.

References 1

api-reference.md2.4 KB

API Reference: Investigating Insider Threat Indicators

Splunk REST API (SIEM Queries)

Endpoint Method Description
/services/search/jobs POST Submit SPL search for user activity timeline
/services/search/jobs/{sid}/results GET Retrieve search results for DLP and access data
/services/saved/searches GET List saved insider threat correlation searches

Microsoft Purview DLP API (Graph)

Endpoint Method Description
/security/alerts_v2 GET Retrieve DLP policy violation alerts
/security/incidents GET List incidents with insider threat classification
/compliance/ediscovery/cases POST Create eDiscovery case for evidence preservation

Microsoft Graph (O365 Activity)

Endpoint Method Description
/auditLogs/signIns GET Query user sign-in logs with location and device
/auditLogs/directoryAudits GET Directory change audit (role assignments, group changes)
/users/{id}/mailFolders/{id}/messages GET Search mailbox for exfiltration via email

Key Libraries

  • splunk-sdk: Python SDK for submitting SPL queries and retrieving results
  • msgraph-sdk: Microsoft Graph API for O365 audit logs and DLP alerts
  • hashlib (stdlib): SHA-256 hashing for chain-of-custody evidence integrity
  • csv (stdlib): Parse exported DLP alert data and access logs

Evidence Handling

Function Purpose
hashlib.sha256() Generate file hash for chain-of-custody log
json.dump() Serialize evidence metadata with timestamps
os.path.getsize() Record evidence file sizes

Configuration

Variable Description
SPLUNK_HOST Splunk search head URL for REST API queries
SPLUNK_TOKEN Bearer token for Splunk authentication
GRAPH_CLIENT_ID Azure AD app registration for Graph API access
GRAPH_TENANT_ID Azure AD tenant ID

References

Scripts 1

agent.py8.7 KB
Display-only source. This catalog never executes bundled scripts.
#!/usr/bin/env python3
"""
Insider Threat Investigation Agent
Automates insider threat indicator collection by correlating SIEM data,
DLP alerts, access logs, and HR events to build investigation timelines.
"""

import csv
import hashlib
import json
import os
import sys
from datetime import datetime, timezone


def load_dlp_alerts(filepath: str) -> list[dict]:
    """Load DLP alert data from exported CSV."""
    alerts = []
    if not os.path.exists(filepath):
        print(f"[!] DLP alerts file not found: {filepath}")
        return alerts
    with open(filepath, "r", newline="") as f:
        reader = csv.DictReader(f)
        for row in reader:
            alerts.append({
                "timestamp": row.get("timestamp", ""),
                "user": row.get("user", ""),
                "policy": row.get("policy_name", ""),
                "action": row.get("action", ""),
                "destination": row.get("destination", ""),
                "file_count": int(row.get("file_count", 0)),
                "bytes_transferred": int(row.get("bytes_transferred", 0)),
                "severity": row.get("severity", "medium"),
            })
    return alerts


def load_access_logs(filepath: str) -> list[dict]:
    """Load authentication and access logs from exported JSON."""
    if not os.path.exists(filepath):
        print(f"[!] Access logs file not found: {filepath}")
        return []
    with open(filepath, "r") as f:
        return json.load(f)


def analyze_data_movement(dlp_alerts: list[dict], subject_user: str) -> dict:
    """Analyze data exfiltration indicators for the subject."""
    user_alerts = [a for a in dlp_alerts if a["user"].lower() == subject_user.lower()]

    total_bytes = sum(a["bytes_transferred"] for a in user_alerts)
    total_files = sum(a["file_count"] for a in user_alerts)
    destinations = {}
    for alert in user_alerts:
        dest = alert["destination"]
        destinations[dest] = destinations.get(dest, 0) + alert["file_count"]

    high_severity = [a for a in user_alerts if a["severity"] == "high"]

    return {
        "total_alerts": len(user_alerts),
        "total_bytes_transferred": total_bytes,
        "total_bytes_gb": round(total_bytes / (1024**3), 2),
        "total_files": total_files,
        "destinations": destinations,
        "high_severity_alerts": len(high_severity),
        "alert_details": user_alerts,
    }


def analyze_access_patterns(access_logs: list[dict], subject_user: str) -> dict:
    """Detect anomalous access patterns for the subject."""
    user_logs = [l for l in access_logs if l.get("user", "").lower() == subject_user.lower()]

    off_hours_events = []
    weekend_events = []
    unique_apps = set()
    unique_ips = set()

    for log in user_logs:
        ts = log.get("timestamp", "")
        try:
            dt = datetime.fromisoformat(ts.replace("Z", "+00:00"))
            hour = dt.hour
            weekday = dt.weekday()
            if hour < 7 or hour > 19:
                off_hours_events.append(log)
            if weekday >= 5:
                weekend_events.append(log)
        except (ValueError, AttributeError):
            pass

        unique_apps.add(log.get("application", "unknown"))
        unique_ips.add(log.get("source_ip", "unknown"))

    return {
        "total_events": len(user_logs),
        "off_hours_events": len(off_hours_events),
        "off_hours_pct": round(len(off_hours_events) / max(len(user_logs), 1) * 100, 1),
        "weekend_events": len(weekend_events),
        "weekend_pct": round(len(weekend_events) / max(len(user_logs), 1) * 100, 1),
        "unique_applications": sorted(unique_apps),
        "unique_source_ips": sorted(unique_ips),
    }


def detect_pre_departure_indicators(
    data_movement: dict, access_patterns: dict, notice_date: str
) -> list[dict]:
    """Identify pre-departure behavioral indicators."""
    indicators = []

    if data_movement["total_bytes_gb"] > 1.0:
        indicators.append({
            "severity": "HIGH",
            "indicator": "Bulk data transfer",
            "detail": f"{data_movement['total_bytes_gb']} GB transferred across {data_movement['total_files']} files",
        })

    if data_movement["high_severity_alerts"] > 0:
        indicators.append({
            "severity": "HIGH",
            "indicator": "High-severity DLP violations",
            "detail": f"{data_movement['high_severity_alerts']} high-severity DLP alerts triggered",
        })

    personal_storage = ["dropbox", "drive.google", "onedrive.live", "mega.nz", "wetransfer"]
    for dest, count in data_movement["destinations"].items():
        if any(ps in dest.lower() for ps in personal_storage):
            indicators.append({
                "severity": "HIGH",
                "indicator": "Transfer to personal cloud storage",
                "detail": f"{count} files sent to {dest}",
            })

    if access_patterns["off_hours_pct"] > 30:
        indicators.append({
            "severity": "MEDIUM",
            "indicator": "Elevated off-hours activity",
            "detail": f"{access_patterns['off_hours_pct']}% of activity outside business hours",
        })

    if access_patterns["weekend_pct"] > 15:
        indicators.append({
            "severity": "MEDIUM",
            "indicator": "Elevated weekend activity",
            "detail": f"{access_patterns['weekend_pct']}% of activity on weekends",
        })

    if len(access_patterns["unique_applications"]) > 15:
        indicators.append({
            "severity": "MEDIUM",
            "indicator": "Broad application access",
            "detail": f"Accessed {len(access_patterns['unique_applications'])} unique applications",
        })

    return indicators


def create_evidence_log(case_id: str, evidence_files: list[str]) -> dict:
    """Create chain-of-custody evidence log with file hashes."""
    items = []
    for filepath in evidence_files:
        if os.path.exists(filepath):
            with open(filepath, "rb") as f:
                content = f.read()
            items.append({
                "item_id": f"EV-{len(items)+1:03d}",
                "file": filepath,
                "sha256": hashlib.sha256(content).hexdigest(),
                "size_bytes": len(content),
                "collected_at": datetime.now(timezone.utc).isoformat(),
            })

    return {
        "case_id": case_id,
        "created_at": datetime.now(timezone.utc).isoformat(),
        "investigator": os.getenv("USER", "soc_analyst"),
        "evidence_items": items,
    }


def generate_report(case_id: str, subject: str, data_mv: dict, access: dict, indicators: list) -> str:
    """Generate insider threat investigation report."""
    lines = [
        f"INSIDER THREAT INVESTIGATION REPORT - {case_id}",
        "=" * 55,
        f"Subject: {subject}",
        f"Report Generated: {datetime.now(timezone.utc).strftime('%Y-%m-%d %H:%M UTC')}",
        "",
        "DATA MOVEMENT ANALYSIS:",
        f"  DLP Alerts: {data_mv['total_alerts']}",
        f"  Data Transferred: {data_mv['total_bytes_gb']} GB ({data_mv['total_files']} files)",
        f"  High-Severity Alerts: {data_mv['high_severity_alerts']}",
        f"  Destinations: {json.dumps(data_mv['destinations'], indent=4)}",
        "",
        "ACCESS PATTERN ANALYSIS:",
        f"  Total Events: {access['total_events']}",
        f"  Off-Hours Activity: {access['off_hours_pct']}%",
        f"  Weekend Activity: {access['weekend_pct']}%",
        f"  Applications Accessed: {len(access['unique_applications'])}",
        "",
        f"INDICATORS IDENTIFIED: {len(indicators)}",
        "-" * 40,
    ]
    for ind in indicators:
        lines.append(f"  [{ind['severity']}] {ind['indicator']}: {ind['detail']}")

    return "\n".join(lines)


if __name__ == "__main__":
    case_id = sys.argv[1] if len(sys.argv) > 1 else "IT-2024-0001"
    subject_user = sys.argv[2] if len(sys.argv) > 2 else "jsmith"
    dlp_file = sys.argv[3] if len(sys.argv) > 3 else "dlp_alerts.csv"
    access_file = sys.argv[4] if len(sys.argv) > 4 else "access_logs.json"

    print(f"[*] Starting insider threat investigation: {case_id}")
    print(f"[*] Subject: {subject_user}")

    dlp_alerts = load_dlp_alerts(dlp_file)
    access_logs = load_access_logs(access_file)

    data_movement = analyze_data_movement(dlp_alerts, subject_user)
    access_patterns = analyze_access_patterns(access_logs, subject_user)
    indicators = detect_pre_departure_indicators(data_movement, access_patterns, "2024-03-15")

    report = generate_report(case_id, subject_user, data_movement, access_patterns, indicators)
    print(report)

    output = f"insider_threat_{case_id}_{datetime.now(timezone.utc).strftime('%Y%m%d')}.json"
    with open(output, "w") as f:
        json.dump({"data_movement": data_movement, "access_patterns": access_patterns, "indicators": indicators}, f, indent=2)
    print(f"\n[*] Results saved to {output}")
Keep exploring