npx skills add mukul975/Anthropic-Cybersecurity-SkillsMITRE ATT&CK
Overview
Spearphishing targets specific individuals using personalized, researched content that bypasses generic spam filters. Email security gateways (SEGs) like Microsoft Defender for Office 365, Proofpoint, Mimecast, and Barracuda provide advanced detection capabilities including behavioral analysis, URL detonation, attachment sandboxing, and impersonation detection. This skill covers configuring these gateways to detect and block targeted phishing attacks.
When to Use
- When investigating security incidents that require detecting spearphishing with email gateway
- When building detection rules or threat hunting queries for this domain
- When SOC analysts need structured procedures for this analysis type
- When validating security monitoring coverage for related attack techniques
Prerequisites
- Access to email security gateway admin console
- Understanding of email flow architecture (MX records, transport rules)
- Familiarity with SPF/DKIM/DMARC authentication
- Knowledge of common spearphishing techniques and pretexts
Key Concepts
Spearphishing Characteristics
- Targeted recipients: Specific individuals, often executives or finance staff
- Researched pretexts: References to real projects, colleagues, or events
- Impersonation: Spoofs trusted senders (CEO, vendor, partner)
- Low volume: Few emails to avoid pattern-based detection
- Urgent tone: Creates pressure to act quickly
Gateway Detection Layers
- Reputation filtering: IP/domain/URL reputation scoring
- Authentication checks: SPF, DKIM, DMARC validation
- Content analysis: NLP-based analysis of email body
- Impersonation detection: Display name and domain similarity matching
- URL analysis: Real-time URL detonation and redirect following
- Attachment sandboxing: Behavioral analysis of attachments in isolated environments
- Behavioral analytics: Anomaly detection in communication patterns
Workflow
Step 1: Configure Impersonation Protection
Microsoft Defender for Office 365:
Security > Anti-phishing policies > Impersonation settings
- Enable user impersonation protection for VIPs
- Enable domain impersonation protection
- Add protected users (CEO, CFO, HR Director)
- Set action: Quarantine message
Proofpoint:
Email Protection > Impostor Classifier
- Enable display name spoofing detection
- Configure lookalike domain detection
- Set Impostor threshold sensitivityStep 2: Configure URL Protection
- Enable Safe Links / URL rewriting
- Enable time-of-click URL detonation
- Block newly registered domains (< 30 days)
- Enable URL redirect chain following
Step 3: Configure Attachment Sandboxing
- Enable Safe Attachments / attachment sandboxing
- Configure dynamic delivery (deliver body, hold attachments)
- Set sandbox detonation timeout to 60+ seconds
- Block macro-enabled Office documents from external senders
Step 4: Create Custom Detection Rules
Use the scripts/process.py to analyze email gateway logs, identify spearphishing patterns, and generate custom detection rules.
Step 5: Configure Alert and Response Actions
- Real-time alerts for impersonation attempts
- Automatic quarantine for high-confidence detections
- User notification with safety tips
- Integration with SIEM for correlation
Tools & Resources
- Microsoft Defender for Office 365: https://security.microsoft.com
- Proofpoint Email Protection: https://www.proofpoint.com/us/products/email-security
- Mimecast Email Security: https://www.mimecast.com/products/email-security/
- Barracuda Email Protection: https://www.barracuda.com/products/email-protection
Validation
- Impersonation protection correctly identifies spoofed VIP display names
- URL detonation catches malicious links in test phishing emails
- Attachment sandboxing detects weaponized documents
- Custom rules trigger on known spearphishing patterns
- SIEM integration receives gateway alerts
References and resources
Everything below is rendered for inspection. Script files are read-only and never run.
References 3
api-reference.md2.2 KB
API Reference: Spearphishing Detection via Email Gateway
Python email Module
Parse EML file
import email
from email import policy
with open("message.eml", "rb") as f:
msg = email.message_from_binary_file(f, policy=policy.default)Security Headers
| Header | Purpose |
|---|---|
Received-SPF |
SPF check result |
Authentication-Results |
SPF, DKIM, DMARC combined |
DKIM-Signature |
DKIM signing info |
ARC-Authentication-Results |
ARC chain results |
X-Mailer |
Client used to send |
Return-Path |
Envelope sender |
Authentication-Results Values
Authentication-Results: mx.google.com;
dkim=pass header.d=example.com;
spf=pass smtp.mailfrom=example.com;
dmarc=passSPF Record Lookup
dig TXT example.com | grep "v=spf1"
# v=spf1 include:_spf.google.com ~allSPF Results
| Result | Meaning |
|---|---|
pass |
Authorized sender |
fail |
Unauthorized (reject) |
softfail |
Unauthorized (accept with mark) |
neutral |
No assertion |
none |
No SPF record |
DKIM Verification
opendkim-testkey -d example.com -s selector -vvvDMARC Record
dig TXT _dmarc.example.com
# v=DMARC1; p=reject; rua=mailto:dmarc@example.comMicrosoft Defender for Office 365 API
Get email threat assessment
POST https://graph.microsoft.com/v1.0/informationProtection/threatAssessmentRequests
Authorization: Bearer {token}
{
"contentType": "mail",
"expectedAssessment": "block",
"category": "phishing",
"mailInfo": {
"internetMessageId": "<message-id>"
}
}Proofpoint TAP API
Get blocked messages
GET https://tap-api-v2.proofpoint.com/v2/siem/messages/blocked
?sinceSeconds=3600
Authorization: Basic {base64_credentials}Response Fields
| Field | Description |
|---|---|
spamScore |
Spam confidence (0-100) |
phishScore |
Phishing confidence (0-100) |
threatsInfoMap |
Threat details array |
fromAddress |
Envelope sender |
Mimecast API — URL Protection
Decode Mimecast URL
POST https://api.mimecast.com/api/ttp/url/decode-url
Authorization: MC {access-key}:{secret-key}
{
"data": [{"url": "https://protect.mimecast.com/..."}]
}standards.md2.4 KB
Standards & References: Detecting Spearphishing with Email Gateway
MITRE ATT&CK References
- T1566.001: Phishing: Spearphishing Attachment
- T1566.002: Phishing: Spearphishing Link
- T1566.003: Phishing: Spearphishing via Service
- T1598.002: Phishing for Information: Spearphishing Attachment
- T1598.003: Phishing for Information: Spearphishing Link
- T1534: Internal Spearphishing
NIST Guidelines
- NIST SP 800-177 Rev.1: Trustworthy Email
- NIST SP 800-53 Rev.5: SI-8 Spam Protection, SI-3 Malicious Code Protection
- NIST CSF: PR.AT (Awareness & Training), DE.CM (Security Continuous Monitoring)
CIS Controls v8
- CIS Control 9: Email and Web Browser Protections
- 9.1: Ensure only approved browsers and email clients are used
- 9.2: Use DNS filtering services
- 9.3: Maintain and enforce network-based URL filters
- 9.6: Block unnecessary file types
- 9.7: Deploy and maintain email server anti-malware protections
Email Gateway Feature Matrix
| Feature | Microsoft Defender | Proofpoint | Mimecast | Barracuda |
|---|---|---|---|---|
| Impersonation detection | Anti-phishing policy | Impostor Classifier | Brand Exploit Protect | Impersonation Protection |
| URL detonation | Safe Links | URL Defense | URL Protect | Link Protection |
| Attachment sandbox | Safe Attachments | Targeted Attack Protection | Attachment Protect | Advanced Threat Protection |
| DMARC enforcement | Built-in | Built-in | DMARC Analyzer | Built-in |
| AI/ML detection | Yes (multiple models) | NexusAI | Yes | Yes |
| User reporting | Report Message add-in | PhishAlarm | Built-in | Phishline |
| SIEM integration | Microsoft Sentinel | Splunk, QRadar | Splunk, Sentinel | Various |
| Auto-remediation (ZAP) | Yes | CLEAR | Yes | Yes |
Detection Indicators for Spearphishing
| Indicator | Weight | Description |
|---|---|---|
| Display name spoofing VIP | High | From name matches protected user but different email |
| Lookalike domain | High | Domain differs by 1-2 characters from legitimate |
| First-time sender to VIP | Medium | No prior communication history |
| Urgency keywords | Medium | "urgent", "immediately", "wire transfer", "confidential" |
| Reply-to mismatch | High | Reply-to differs from From address |
| External sender with internal branding | High | Email mimics internal templates |
| Newly registered domain | High | Sending domain < 30 days old |
| Authentication failure | High | SPF/DKIM/DMARC fail |
workflows.md3.7 KB
Workflows: Detecting Spearphishing with Email Gateway
Workflow 1: Multi-Layer Detection Pipeline
Inbound Email Arrives at Gateway
|
v
[Layer 1: Connection Filtering]
+-- Check sender IP reputation
+-- Check RBL/DNSBL blacklists
+-- Rate limiting / throttling
|
v
[Layer 2: Authentication]
+-- Verify SPF alignment
+-- Verify DKIM signature
+-- Evaluate DMARC policy
+-- Check ARC headers (forwarded mail)
|
v
[Layer 3: Impersonation Detection]
+-- Compare display name against VIP list
+-- Check domain similarity (Levenshtein distance)
+-- Evaluate sender reputation
+-- First-time sender analysis
|
v
[Layer 4: Content Analysis]
+-- NLP analysis for urgency/social engineering
+-- Business context anomaly detection
+-- Keyword pattern matching
+-- Language analysis
|
v
[Layer 5: URL Analysis]
+-- URL reputation check
+-- Domain age verification
+-- Real-time URL detonation
+-- Redirect chain following
+-- Visual similarity to legitimate sites
|
v
[Layer 6: Attachment Analysis]
+-- File type validation
+-- Sandbox detonation
+-- Macro analysis
+-- Embedded object detection
|
v
[Decision Engine]
+-- Aggregate scores from all layers
+-- Apply organizational policy
|
+-- DELIVER: Low risk
+-- TAG: Add warning banner
+-- QUARANTINE: Moderate risk
+-- BLOCK: High risk, drop messageWorkflow 2: VIP Impersonation Detection
Email arrives with From display name matching VIP list
|
v
[Check: Is sending domain authorized for this VIP?]
|
+-- YES: Check DKIM/SPF --> If pass, deliver normally
|
+-- NO: Impersonation suspected
|
v
[Calculate domain similarity score]
|
+-- Exact match (different email): CRITICAL - Block
+-- Lookalike domain (1-2 char diff): HIGH - Quarantine
+-- Similar but different: MEDIUM - Tag with warning
|
v
[Additional checks]
+-- Has this sender emailed before?
+-- Is the sending infrastructure legitimate?
+-- Does email content match typical VIP communication?
|
v
[Action: Quarantine + Alert SOC + Notify recipient manager]Workflow 3: Spearphishing Response
Gateway detects potential spearphishing
|
v
[Automated Response]
+-- Quarantine message
+-- Generate alert in SIEM
+-- Extract IOCs (sender, domain, URLs, hashes)
|
v
[SOC Analyst Review]
+-- Review quarantined message
+-- Analyze full headers
+-- Investigate sending infrastructure
+-- Check if other users received similar emails
|
+-- FALSE POSITIVE
| +-- Release from quarantine
| +-- Whitelist if legitimate
| +-- Update detection rules
|
+-- CONFIRMED SPEARPHISHING
+-- Block sender domain organization-wide
+-- Search mailboxes for similar messages (retroactive)
+-- Auto-purge any delivered copies (ZAP)
+-- Notify targeted users
+-- Submit IOCs to threat intelligence
+-- Check for any successful credential compromise
+-- Update VIP protection list if neededWorkflow 4: Gateway Tuning Cycle
Monthly Review
|
+-- Pull detection statistics from gateway
+-- Analyze false positive rate
+-- Analyze false negative rate (user-reported misses)
+-- Review quarantine volumes
|
v
[Identify gaps]
+-- New impersonation patterns?
+-- New sending domains to whitelist/blacklist?
+-- Policy thresholds too aggressive/permissive?
|
v
[Adjust configuration]
+-- Update VIP protection list (new hires, departures)
+-- Tune sensitivity thresholds
+-- Add custom transport rules
+-- Update URL/domain blocklists
|
v
[Validate changes]
+-- Send test phishing emails
+-- Verify legitimate mail still flows
+-- Document changesScripts 2
agent.py5.6 KB
#!/usr/bin/env python3
"""Agent for detecting spearphishing emails using email gateway log analysis."""
import argparse
import email
import json
import os
import re
from datetime import datetime, timezone
from email import policy
EXECUTIVE_TITLES = [
"ceo", "cfo", "cto", "cio", "coo", "president", "director",
"vp", "vice president", "managing director", "partner",
]
URGENCY_KEYWORDS = [
"urgent", "immediate", "asap", "right away", "time sensitive",
"confidential", "do not share", "wire transfer", "bank account",
"update payment", "invoice attached", "past due",
]
SUSPICIOUS_EXTENSIONS = {
".exe", ".scr", ".bat", ".cmd", ".ps1", ".vbs", ".js",
".hta", ".lnk", ".iso", ".img", ".dll", ".msi",
}
def parse_email_headers(eml_path):
"""Extract security-relevant headers from an email."""
with open(eml_path, "rb") as f:
msg = email.message_from_binary_file(f, policy=policy.default)
headers = {
"from": msg.get("From", ""),
"to": msg.get("To", ""),
"subject": msg.get("Subject", ""),
"reply_to": msg.get("Reply-To", ""),
"return_path": msg.get("Return-Path", ""),
"received_spf": msg.get("Received-SPF", ""),
"dkim_signature": msg.get("DKIM-Signature", ""),
"authentication_results": msg.get("Authentication-Results", ""),
"x_mailer": msg.get("X-Mailer", ""),
"message_id": msg.get("Message-ID", ""),
}
return headers, msg
def check_authentication(headers):
"""Verify SPF, DKIM, DMARC authentication results."""
issues = []
auth_results = headers.get("authentication_results", "").lower()
if "spf=fail" in auth_results or "spf=softfail" in auth_results:
issues.append("SPF validation failed")
if "dkim=fail" in auth_results:
issues.append("DKIM validation failed")
if "dmarc=fail" in auth_results:
issues.append("DMARC validation failed")
if not headers.get("dkim_signature"):
issues.append("No DKIM signature present")
from_domain = re.search(r"@([\w.-]+)", headers.get("from", ""))
reply_domain = re.search(r"@([\w.-]+)", headers.get("reply_to", ""))
if from_domain and reply_domain and from_domain.group(1) != reply_domain.group(1):
issues.append(f"Reply-To domain mismatch: {reply_domain.group(1)} vs {from_domain.group(1)}")
return issues
def check_content_indicators(msg):
"""Analyze email body for spearphishing content indicators."""
indicators = []
body = ""
for part in msg.walk():
if part.get_content_type() in ("text/plain", "text/html"):
payload = part.get_payload(decode=True)
if payload:
body += payload.decode("utf-8", errors="replace")
body_lower = body.lower()
for kw in URGENCY_KEYWORDS:
if kw in body_lower:
indicators.append(f"Urgency keyword: '{kw}'")
urls = re.findall(r'https?://[^\s<>"\']+', body)
for url in urls[:10]:
if re.search(r'\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}', url):
indicators.append(f"URL with IP address: {url[:80]}")
if len(url) > 100:
indicators.append(f"Unusually long URL: {url[:80]}...")
attachments = []
for part in msg.walk():
fname = part.get_filename()
if fname:
ext = os.path.splitext(fname)[1].lower()
attachments.append(fname)
if ext in SUSPICIOUS_EXTENSIONS:
indicators.append(f"Suspicious attachment: {fname}")
if ".." in fname or ext == ".exe.pdf":
indicators.append(f"Double extension: {fname}")
return indicators, attachments
def analyze_email(eml_path):
"""Full spearphishing analysis of a single email."""
headers, msg = parse_email_headers(eml_path)
auth_issues = check_authentication(headers)
content_indicators, attachments = check_content_indicators(msg)
all_indicators = auth_issues + content_indicators
score = len(all_indicators) * 15
risk = "CRITICAL" if score >= 75 else "HIGH" if score >= 50 else "MEDIUM" if score >= 25 else "LOW"
return {
"file": eml_path,
"from": headers["from"],
"to": headers["to"],
"subject": headers["subject"],
"auth_issues": auth_issues,
"content_indicators": content_indicators,
"attachments": attachments,
"risk_score": min(score, 100),
"risk_level": risk,
}
def main():
parser = argparse.ArgumentParser(
description="Detect spearphishing via email gateway analysis"
)
parser.add_argument("input", help=".eml file or directory")
parser.add_argument("--output", "-o", help="Output JSON report")
parser.add_argument("--verbose", "-v", action="store_true")
args = parser.parse_args()
print("[*] Spearphishing Detection Agent")
results = []
if os.path.isdir(args.input):
for root, _, files in os.walk(args.input):
for f in files:
if f.lower().endswith(".eml"):
results.append(analyze_email(os.path.join(root, f)))
else:
results.append(analyze_email(args.input))
flagged = [r for r in results if r["risk_level"] in ("HIGH", "CRITICAL")]
report = {
"timestamp": datetime.now(timezone.utc).isoformat(),
"emails_scanned": len(results),
"spearphishing_detected": len(flagged),
"results": results,
}
print(f"[*] Scanned {len(results)} emails, {len(flagged)} flagged")
if args.output:
with open(args.output, "w") as f:
json.dump(report, f, indent=2)
print(f"[*] Report saved to {args.output}")
else:
print(json.dumps(report, indent=2))
if __name__ == "__main__":
main()
process.py21.1 KB
#!/usr/bin/env python3
"""
Spearphishing Detection Engine for Email Gateway Logs
Analyzes email gateway logs to detect spearphishing patterns including
impersonation, lookalike domains, and behavioral anomalies. Generates
custom detection rules and threat reports.
Usage:
python process.py analyze --log-file gateway_log.json
python process.py detect --email-file email.eml
python process.py rules --output detection_rules.yaml
"""
import argparse
import json
import re
import sys
import hashlib
from datetime import datetime, timezone
from pathlib import Path
from dataclasses import dataclass, field, asdict
from typing import Optional
from collections import defaultdict
try:
import requests
HAS_REQUESTS = True
except ImportError:
HAS_REQUESTS = False
@dataclass
class SpearphishingIndicator:
"""A detected spearphishing indicator."""
indicator_type: str = ""
description: str = ""
severity: str = "medium"
confidence: float = 0.0
raw_value: str = ""
mitre_technique: str = ""
@dataclass
class DomainSimilarity:
"""Domain similarity analysis result."""
original_domain: str = ""
suspicious_domain: str = ""
distance: int = 0
technique: str = ""
confidence: float = 0.0
@dataclass
class EmailAnalysis:
"""Complete spearphishing analysis for a single email."""
message_id: str = ""
from_address: str = ""
from_display_name: str = ""
from_domain: str = ""
to_address: str = ""
subject: str = ""
date: str = ""
indicators: list = field(default_factory=list)
risk_score: float = 0.0
risk_level: str = "low"
domain_similarities: list = field(default_factory=list)
is_spearphishing: bool = False
# VIP list for impersonation detection (configure per organization)
DEFAULT_VIP_NAMES = [
"CEO", "CFO", "CTO", "CISO", "COO",
"Chief Executive", "Chief Financial", "Chief Technology",
"President", "Vice President", "Director",
]
# Common urgency keywords in spearphishing
URGENCY_KEYWORDS = [
r'\burgent\b', r'\bimmediately\b', r'\basap\b', r'\btime.?sensitive\b',
r'\bwire\s+transfer\b', r'\bbank\s+transfer\b', r'\bgift\s+card\b',
r'\bconfidential\b', r'\bdo\s+not\s+share\b', r'\bsecret\b',
r'\bpayment\b', r'\binvoice\b', r'\boverdue\b', r'\bfinal\s+notice\b',
r'\baccount\s+suspen', r'\bverify\s+your\b', r'\bconfirm\s+your\b',
r'\bunusual\s+activity\b', r'\bsecurity\s+alert\b',
]
# Legitimate domains for similarity comparison
COMMON_TARGETS = [
"microsoft.com", "google.com", "apple.com", "amazon.com",
"paypal.com", "netflix.com", "linkedin.com", "facebook.com",
"dropbox.com", "docusign.com", "zoom.us", "slack.com",
"office365.com", "outlook.com", "gmail.com",
]
def levenshtein_distance(s1: str, s2: str) -> int:
"""Calculate Levenshtein edit distance between two strings."""
if len(s1) < len(s2):
return levenshtein_distance(s2, s1)
if len(s2) == 0:
return len(s1)
prev_row = range(len(s2) + 1)
for i, c1 in enumerate(s1):
curr_row = [i + 1]
for j, c2 in enumerate(s2):
insertions = prev_row[j + 1] + 1
deletions = curr_row[j] + 1
substitutions = prev_row[j] + (c1 != c2)
curr_row.append(min(insertions, deletions, substitutions))
prev_row = curr_row
return prev_row[-1]
def detect_homograph(domain: str) -> list:
"""Detect IDN homograph attacks in domain names."""
homograph_map = {
'a': ['@', 'а', 'ɑ'], # Cyrillic а, Latin alpha
'e': ['е', 'ё', 'э'], # Cyrillic variants
'o': ['о', '0', 'ο'], # Cyrillic о, zero, Greek omicron
'p': ['р', 'ρ'], # Cyrillic р, Greek rho
'c': ['с', 'ç'], # Cyrillic с
'x': ['х', 'χ'], # Cyrillic х, Greek chi
'y': ['у', 'γ'], # Cyrillic у
'i': ['і', 'і', '1', 'l'],
}
findings = []
for i, char in enumerate(domain):
for latin, lookalikes in homograph_map.items():
if char in lookalikes:
findings.append({
"position": i,
"character": char,
"looks_like": latin,
"type": "homograph"
})
return findings
def check_domain_similarity(domain: str, known_domains: list = None) -> list:
"""Check if a domain is similar to known legitimate domains."""
if known_domains is None:
known_domains = COMMON_TARGETS
similarities = []
domain_base = domain.split(".")[0] if "." in domain else domain
for known in known_domains:
known_base = known.split(".")[0]
dist = levenshtein_distance(domain_base.lower(), known_base.lower())
if 0 < dist <= 2:
technique = "typosquatting"
if len(domain_base) != len(known_base):
technique = "character_addition" if len(domain_base) > len(known_base) else "character_omission"
elif dist == 1:
for i, (a, b) in enumerate(zip(domain_base, known_base)):
if a != b:
if i > 0 and domain_base[i-1:i+1] == known_base[i:i-2:-1] if i > 0 else False:
technique = "transposition"
break
confidence = max(0, 1.0 - dist * 0.3)
similarities.append(DomainSimilarity(
original_domain=known,
suspicious_domain=domain,
distance=dist,
technique=technique,
confidence=confidence
))
# Check for homographs
homographs = detect_homograph(domain)
if homographs:
similarities.append(DomainSimilarity(
original_domain="(homograph detection)",
suspicious_domain=domain,
distance=0,
technique="homograph",
confidence=0.9
))
return sorted(similarities, key=lambda x: x.distance)
def detect_urgency(text: str) -> list:
"""Detect urgency patterns in email text."""
findings = []
text_lower = text.lower()
for pattern in URGENCY_KEYWORDS:
matches = re.findall(pattern, text_lower, re.IGNORECASE)
if matches:
findings.append({
"pattern": pattern,
"matches": matches,
"count": len(matches)
})
return findings
def detect_impersonation(display_name: str, vip_names: list = None) -> list:
"""Check if display name impersonates a VIP."""
if vip_names is None:
vip_names = DEFAULT_VIP_NAMES
findings = []
name_lower = display_name.lower()
for vip in vip_names:
if vip.lower() in name_lower:
findings.append({
"matched_vip": vip,
"display_name": display_name,
"confidence": 0.8
})
return findings
def analyze_email(headers: dict, body: str = "",
vip_names: list = None,
known_domains: list = None) -> EmailAnalysis:
"""Analyze a single email for spearphishing indicators."""
analysis = EmailAnalysis()
analysis.from_address = headers.get("from", "")
analysis.from_display_name = headers.get("from_display_name", "")
analysis.from_domain = headers.get("from_domain", "")
analysis.to_address = headers.get("to", "")
analysis.subject = headers.get("subject", "")
analysis.date = headers.get("date", "")
analysis.message_id = headers.get("message_id", "")
# Extract domain from from_address if not provided
if not analysis.from_domain and analysis.from_address:
match = re.search(r'@([\w.-]+)', analysis.from_address)
if match:
analysis.from_domain = match.group(1).lower()
# Extract display name if not provided
if not analysis.from_display_name and analysis.from_address:
match = re.match(r'"?([^"<]+)"?\s*<', analysis.from_address)
if match:
analysis.from_display_name = match.group(1).strip()
score = 0.0
# Check 1: Domain similarity
if analysis.from_domain:
similarities = check_domain_similarity(analysis.from_domain, known_domains)
analysis.domain_similarities = similarities
for sim in similarities:
if sim.distance <= 1:
analysis.indicators.append(SpearphishingIndicator(
indicator_type="lookalike_domain",
description=f"Domain '{sim.suspicious_domain}' is {sim.distance} edit(s) "
f"from '{sim.original_domain}' ({sim.technique})",
severity="critical" if sim.distance == 1 else "high",
confidence=sim.confidence,
raw_value=sim.suspicious_domain,
mitre_technique="T1566.002"
))
score += 30 * sim.confidence
elif sim.distance == 2:
analysis.indicators.append(SpearphishingIndicator(
indicator_type="similar_domain",
description=f"Domain '{sim.suspicious_domain}' resembles "
f"'{sim.original_domain}' (distance={sim.distance})",
severity="medium",
confidence=sim.confidence,
raw_value=sim.suspicious_domain,
mitre_technique="T1566.002"
))
score += 15 * sim.confidence
# Check 2: VIP impersonation
if analysis.from_display_name:
impersonations = detect_impersonation(analysis.from_display_name, vip_names)
for imp in impersonations:
analysis.indicators.append(SpearphishingIndicator(
indicator_type="vip_impersonation",
description=f"Display name '{imp['display_name']}' matches VIP "
f"keyword '{imp['matched_vip']}'",
severity="high",
confidence=imp["confidence"],
raw_value=analysis.from_display_name,
mitre_technique="T1566.001"
))
score += 25 * imp["confidence"]
# Check 3: Urgency indicators in subject
urgency_subject = detect_urgency(analysis.subject)
if urgency_subject:
analysis.indicators.append(SpearphishingIndicator(
indicator_type="urgency_subject",
description=f"Subject contains {len(urgency_subject)} urgency pattern(s)",
severity="medium",
confidence=min(len(urgency_subject) * 0.3, 0.9),
raw_value=analysis.subject,
mitre_technique="T1566"
))
score += min(len(urgency_subject) * 5, 20)
# Check 4: Urgency indicators in body
if body:
urgency_body = detect_urgency(body)
if urgency_body:
total_matches = sum(u["count"] for u in urgency_body)
analysis.indicators.append(SpearphishingIndicator(
indicator_type="urgency_body",
description=f"Body contains {total_matches} urgency keyword(s) "
f"across {len(urgency_body)} pattern(s)",
severity="medium",
confidence=min(total_matches * 0.15, 0.9),
raw_value=f"{total_matches} matches",
mitre_technique="T1566"
))
score += min(total_matches * 3, 15)
# Check 5: Authentication failures
auth_results = headers.get("authentication_results", "")
if auth_results:
if "spf=fail" in auth_results.lower() or "spf=softfail" in auth_results.lower():
analysis.indicators.append(SpearphishingIndicator(
indicator_type="spf_failure",
description="SPF authentication failed",
severity="high",
confidence=0.7,
raw_value=auth_results,
mitre_technique="T1566"
))
score += 20
if "dkim=fail" in auth_results.lower():
analysis.indicators.append(SpearphishingIndicator(
indicator_type="dkim_failure",
description="DKIM authentication failed",
severity="high",
confidence=0.7,
raw_value=auth_results,
mitre_technique="T1566"
))
score += 20
if "dmarc=fail" in auth_results.lower():
analysis.indicators.append(SpearphishingIndicator(
indicator_type="dmarc_failure",
description="DMARC authentication failed",
severity="critical",
confidence=0.8,
raw_value=auth_results,
mitre_technique="T1566"
))
score += 25
# Check 6: Reply-to mismatch
reply_to = headers.get("reply_to", "")
if reply_to and analysis.from_address:
reply_domain = ""
match = re.search(r'@([\w.-]+)', reply_to)
if match:
reply_domain = match.group(1).lower()
if reply_domain and reply_domain != analysis.from_domain:
analysis.indicators.append(SpearphishingIndicator(
indicator_type="reply_to_mismatch",
description=f"Reply-To domain ({reply_domain}) differs from "
f"From domain ({analysis.from_domain})",
severity="high",
confidence=0.85,
raw_value=f"From: {analysis.from_domain}, Reply-To: {reply_domain}",
mitre_technique="T1566"
))
score += 20
# Calculate final risk
analysis.risk_score = min(score, 100)
if analysis.risk_score >= 70:
analysis.risk_level = "critical"
analysis.is_spearphishing = True
elif analysis.risk_score >= 50:
analysis.risk_level = "high"
analysis.is_spearphishing = True
elif analysis.risk_score >= 30:
analysis.risk_level = "medium"
elif analysis.risk_score >= 10:
analysis.risk_level = "low"
else:
analysis.risk_level = "clean"
return analysis
def generate_detection_rules(indicators_db: list) -> str:
"""Generate YAML detection rules from accumulated indicators."""
rules = []
# Group by indicator type
by_type = defaultdict(list)
for ind in indicators_db:
by_type[ind["indicator_type"]].append(ind)
rule_id = 1
for ind_type, indicators in by_type.items():
values = list(set(ind.get("raw_value", "") for ind in indicators if ind.get("raw_value")))
if not values:
continue
rule = {
"id": f"SPEAR-{rule_id:04d}",
"name": f"Spearphishing {ind_type.replace('_', ' ').title()} Detection",
"type": ind_type,
"severity": indicators[0].get("severity", "medium"),
"mitre": indicators[0].get("mitre_technique", "T1566"),
"description": indicators[0].get("description", ""),
"action": "quarantine" if indicators[0].get("severity") in ("high", "critical") else "tag",
"values_count": len(values),
"sample_values": values[:5]
}
rules.append(rule)
rule_id += 1
# Format as YAML-like output
output_lines = ["# Auto-generated Spearphishing Detection Rules",
f"# Generated: {datetime.now(timezone.utc).isoformat()}",
f"# Total rules: {len(rules)}", ""]
for rule in rules:
output_lines.append(f"- id: {rule['id']}")
output_lines.append(f" name: \"{rule['name']}\"")
output_lines.append(f" type: {rule['type']}")
output_lines.append(f" severity: {rule['severity']}")
output_lines.append(f" mitre: {rule['mitre']}")
output_lines.append(f" action: {rule['action']}")
output_lines.append(f" indicators_count: {rule['values_count']}")
output_lines.append(f" sample_values:")
for val in rule["sample_values"]:
output_lines.append(f" - \"{val}\"")
output_lines.append("")
return "\n".join(output_lines)
def format_analysis_report(analysis: EmailAnalysis) -> str:
"""Format analysis as text report."""
lines = []
lines.append("=" * 60)
lines.append(" SPEARPHISHING DETECTION REPORT")
lines.append("=" * 60)
lines.append(f" Risk Level: {analysis.risk_level.upper()} "
f"(Score: {analysis.risk_score:.0f}/100)")
lines.append(f" Verdict: {'SPEARPHISHING DETECTED' if analysis.is_spearphishing else 'NOT DETECTED'}")
lines.append("")
lines.append(f" From: {analysis.from_display_name} <{analysis.from_address}>")
lines.append(f" To: {analysis.to_address}")
lines.append(f" Subject: {analysis.subject}")
lines.append(f" Date: {analysis.date}")
lines.append("")
if analysis.indicators:
lines.append(f"[INDICATORS] ({len(analysis.indicators)} found)")
for i, ind in enumerate(analysis.indicators, 1):
lines.append(f" {i}. [{ind.severity.upper()}] {ind.description}")
lines.append(f" Type: {ind.indicator_type} | "
f"MITRE: {ind.mitre_technique} | "
f"Confidence: {ind.confidence:.0%}")
else:
lines.append("[INDICATORS] None found")
if analysis.domain_similarities:
lines.append(f"\n[DOMAIN ANALYSIS]")
for sim in analysis.domain_similarities:
lines.append(f" {sim.suspicious_domain} ~ {sim.original_domain} "
f"(distance={sim.distance}, technique={sim.technique})")
lines.append("=" * 60)
return "\n".join(lines)
def main():
parser = argparse.ArgumentParser(description="Spearphishing Detection Engine")
subparsers = parser.add_subparsers(dest="command")
analyze_parser = subparsers.add_parser("analyze", help="Analyze gateway log file")
analyze_parser.add_argument("--log-file", required=True, help="JSON log file")
analyze_parser.add_argument("--output", "-o", help="Output file")
detect_parser = subparsers.add_parser("detect", help="Detect spearphishing in single email")
detect_parser.add_argument("--from", dest="from_addr", required=True)
detect_parser.add_argument("--from-name", default="")
detect_parser.add_argument("--to", dest="to_addr", default="")
detect_parser.add_argument("--subject", default="")
detect_parser.add_argument("--body", default="")
detect_parser.add_argument("--auth-results", default="")
domain_parser = subparsers.add_parser("check-domain", help="Check domain similarity")
domain_parser.add_argument("domain", help="Domain to check")
rules_parser = subparsers.add_parser("rules", help="Generate detection rules from log")
rules_parser.add_argument("--log-file", required=True)
rules_parser.add_argument("--output", "-o", help="Output rules file")
parser.add_argument("--json", action="store_true", help="Output as JSON")
args = parser.parse_args()
if args.command == "analyze":
with open(args.log_file, "r") as f:
log_entries = json.load(f)
results = []
for entry in log_entries:
headers = entry.get("headers", entry)
body = entry.get("body", "")
analysis = analyze_email(headers, body)
results.append(analysis)
spearphishing_count = sum(1 for r in results if r.is_spearphishing)
print(f"Analyzed {len(results)} emails, detected {spearphishing_count} spearphishing attempts")
print()
for analysis in results:
if analysis.is_spearphishing:
if args.json:
print(json.dumps(asdict(analysis), indent=2, default=str))
else:
print(format_analysis_report(analysis))
print()
elif args.command == "detect":
headers = {
"from": args.from_addr,
"from_display_name": args.from_name,
"to": args.to_addr,
"subject": args.subject,
"authentication_results": args.auth_results,
}
analysis = analyze_email(headers, args.body)
if args.json:
print(json.dumps(asdict(analysis), indent=2, default=str))
else:
print(format_analysis_report(analysis))
elif args.command == "check-domain":
similarities = check_domain_similarity(args.domain)
if similarities:
print(f"Domain '{args.domain}' is similar to:")
for sim in similarities:
print(f" - {sim.original_domain} (distance={sim.distance}, "
f"technique={sim.technique}, confidence={sim.confidence:.0%})")
else:
print(f"Domain '{args.domain}' has no known similarities")
elif args.command == "rules":
with open(args.log_file, "r") as f:
log_entries = json.load(f)
all_indicators = []
for entry in log_entries:
analysis = analyze_email(entry.get("headers", entry), entry.get("body", ""))
for ind in analysis.indicators:
all_indicators.append(asdict(ind))
rules = generate_detection_rules(all_indicators)
if args.output:
with open(args.output, "w") as f:
f.write(rules)
print(f"Rules written to {args.output}")
else:
print(rules)
else:
parser.print_help()
if __name__ == "__main__":
main()