ransomware defense

Recovering from Ransomware Attack

Executes structured recovery from a ransomware incident following NIST and CISA frameworks, including environment isolation, forensic evidence preservation, clean infrastructure rebuild, prioritized system restoration from verified backups, credential reset, and validation against re-infection. Covers Active Directory recovery, database restoration, and application stack rebuild in dependency order. Activates for requests involving ransomware recovery, post-encryption restoration, or disaster recovery from ransomware.

backupdefenseincident-responseransomwarerecovery
Install this skill
npx skills add mukul975/Anthropic-Cybersecurity-Skills
Framework mappings

When to Use

  • After ransomware has encrypted production systems and the decision has been made to recover from backups
  • When building or validating a ransomware recovery runbook before an actual incident
  • After receiving a decryption key (paid ransom or law enforcement provided) and needing to safely decrypt
  • When partial recovery is needed alongside decryption of remaining systems
  • Conducting a recovery drill to validate RTO commitments

Do not use before completing containment and forensic scoping. Premature recovery without understanding the attacker's access and persistence mechanisms risks re-infection.

Prerequisites

  • Incident declared and containment phase completed (all attacker access severed)
  • Forensic evidence preserved (disk images, memory dumps, network captures)
  • Backup integrity verified (immutable/air-gapped copies confirmed clean)
  • Clean build media available (OS installation media, golden images)
  • Recovery environment prepared (clean network segment isolated from compromised infrastructure)
  • Recovery priority list documented (Tier 1/2/3 systems in dependency order)

Workflow

Step 1: Establish Clean Recovery Environment

Build recovery infrastructure isolated from the compromised network:

# Create isolated recovery VLAN
# No connectivity to compromised network segments
# Dedicated internet access for patch downloads only (via proxy)
 
# Recovery network architecture:
# VLAN 999 (Recovery) - 10.99.0.0/24
#   - Recovery workstations (10.99.0.10-20)
#   - Recovered DCs (10.99.0.50-55)
#   - Recovered servers (10.99.0.100+)
#   - Proxy for internet (10.99.0.1) - patches and updates only
 
# Firewall rules: DENY all from recovery VLAN to production VLANs
# Allow: Recovery VLAN -> Internet (HTTPS only, via proxy)
# Allow: Recovery VLAN -> Backup infrastructure (restore traffic only)

Step 2: Recover Identity Infrastructure First

Active Directory must be recovered before any domain-joined systems:

# AD Recovery Procedure
# Step 2a: Restore AD from known-good backup
# Use DSRM (Directory Services Restore Mode) boot
 
# 1. Build clean Windows Server from ISO
# 2. Promote as DC using AD restore
# 3. Restore System State from immutable backup
 
# Verify AD backup is pre-compromise
# Check backup timestamp against earliest known compromise date
wbadmin get versions -backuptarget:E: -machine:DC01
 
# Restore system state in DSRM
wbadmin start systemstaterecovery -version:02/15/2026-04:00 -backuptarget:E: -machine:DC01 -quiet
 
# After restore, reset critical accounts
# Reset krbtgt password TWICE (invalidates all Kerberos tickets)
# This prevents Golden Ticket persistence
Import-Module ActiveDirectory
Set-ADAccountPassword -Identity krbtgt -Reset -NewPassword (ConvertTo-SecureString "NewKrbtgt2026!Complex#1" -AsPlainText -Force)
# Wait for replication (minimum 12 hours), then reset again
Set-ADAccountPassword -Identity krbtgt -Reset -NewPassword (ConvertTo-SecureString "NewKrbtgt2026!Complex#2" -AsPlainText -Force)
 
# Reset all privileged account passwords
$privilegedGroups = @("Domain Admins", "Enterprise Admins", "Schema Admins", "Administrators")
foreach ($group in $privilegedGroups) {
    Get-ADGroupMember -Identity $group -Recursive | ForEach-Object {
        Set-ADAccountPassword -Identity $_.SamAccountName -Reset `
            -NewPassword (ConvertTo-SecureString (New-Guid).Guid -AsPlainText -Force)
        Set-ADUser -Identity $_.SamAccountName -ChangePasswordAtLogon $true
    }
}
 
# Validate AD health
dcdiag /v /c /d /e /s:DC01
repadmin /showrepl

Step 3: Validate Backup Integrity Before Restoration

# Scan backup files for ransomware artifacts before restoring
# Use offline antivirus scanning on backup mount
 
# Mount backup as read-only
mount -o ro,noexec /dev/backup_lv /mnt/backup_verify
 
# Scan with ClamAV
clamscan -r --infected --log=/var/log/backup_scan.log /mnt/backup_verify
 
# Check for known ransomware indicators
find /mnt/backup_verify -name "*.encrypted" -o -name "*.locked" \
    -o -name "*.lockbit" -o -name "DECRYPT_*" -o -name "readme.txt" \
    -o -name "RECOVER-*" -o -name "HOW_TO_*" | tee /var/log/ransomware_check.log
 
# Verify database consistency (SQL Server example)
# Restore database to temporary instance for validation
RESTORE VERIFYONLY FROM DISK = '/mnt/backup_verify/databases/erp_db.bak'
    WITH CHECKSUM

Step 4: Restore Systems in Priority Order

Follow dependency-based recovery sequence:

Recovery Order:
Phase 1 (Hours 0-4): Identity & Infrastructure
  1. Domain Controllers (AD, DNS, DHCP)
  2. Certificate Authority (if applicable)
  3. Core network services (DHCP, NTP)
 
Phase 2 (Hours 4-12): Critical Business Systems
  4. Database servers (SQL, Oracle, PostgreSQL)
  5. Core business applications (ERP, CRM)
  6. Email (Exchange, M365 hybrid)
 
Phase 3 (Hours 12-24): Important Systems
  7. File servers
  8. Web applications
  9. Monitoring and security tools (SIEM, EDR)
 
Phase 4 (Hours 24-48): Remaining Systems
  10. Development environments
  11. Archive systems
  12. Non-critical applications
# Veeam Instant Recovery - fastest restore for VMware/Hyper-V
# Boots VM directly from backup file, then migrates to production storage
 
# Instant recovery for Tier 1 system
Start-VBRInstantRecovery -RestorePoint (Get-VBRRestorePoint -Name "DC01" |
    Sort-Object CreationTime -Descending | Select-Object -First 1) `
    -VMName "DC01-Recovered" `
    -Server (Get-VBRServer -Name "esxi01.recovery.local") `
    -Datastore "recovery-datastore"
 
# After validation, migrate to production storage
Start-VBRQuickMigration -VM "DC01-Recovered" `
    -Server (Get-VBRServer -Name "esxi01.prod.local") `
    -Datastore "production-datastore"

Step 5: Validate Recovered Systems and Harden

Before connecting recovered systems to production:

# Check for persistence mechanisms
# Scheduled Tasks
Get-ScheduledTask | Where-Object {$_.State -ne "Disabled"} |
    Select-Object TaskName, TaskPath, State, Author |
    Export-Csv C:\recovery\scheduled_tasks.csv
 
# Services
Get-Service | Where-Object {$_.StartType -eq "Automatic"} |
    Select-Object Name, DisplayName, StartType, Status |
    Export-Csv C:\recovery\auto_services.csv
 
# Startup items
Get-CimInstance Win32_StartupCommand |
    Select-Object Name, Command, Location, User |
    Export-Csv C:\recovery\startup_items.csv
 
# WMI event subscriptions (common persistence)
Get-WmiObject -Namespace root\subscription -Class __EventFilter
Get-WmiObject -Namespace root\subscription -Class __EventConsumer
 
# Registry run keys
Get-ItemProperty "HKLM:\Software\Microsoft\Windows\CurrentVersion\Run"
Get-ItemProperty "HKLM:\Software\Microsoft\Windows\CurrentVersion\RunOnce"
Get-ItemProperty "HKCU:\Software\Microsoft\Windows\CurrentVersion\Run"
 
# Verify no unauthorized admin accounts
Get-LocalGroupMember -Group "Administrators"
Get-ADGroupMember -Identity "Domain Admins"
 
# Apply latest patches before connecting to production
Install-WindowsUpdate -AcceptAll -AutoReboot

Step 6: Phased Network Reconnection

Phase 1: Reconnect identity infrastructure
  - DCs online in production VLAN
  - Validate replication and authentication
  - Monitor for suspicious authentication patterns
 
Phase 2: Reconnect Tier 1 systems
  - One system at a time
  - Monitor EDR for 1 hour before proceeding to next
  - Validate application functionality
 
Phase 3: Reconnect remaining systems
  - Groups of 5-10 systems
  - Continue monitoring for re-infection indicators
 
Throughout: SOC monitoring on high alert
  - EDR in aggressive blocking mode
  - All previous IOCs loaded in detection rules
  - Canary files deployed on recovered systems

Key Concepts

Term Definition
DSRM Directory Services Restore Mode: special boot mode for domain controllers that allows AD database restoration
krbtgt Reset Resetting the krbtgt account password twice invalidates all Kerberos tickets, defeating Golden Ticket persistence
Instant Recovery Backup technology that boots a VM directly from backup storage for immediate availability while migrating data in background
Evidence Preservation Maintaining forensic images and logs before recovery begins, required for law enforcement and insurance claims
Clean Build Rebuilding systems from trusted installation media rather than attempting to clean infected systems
Dependency Chain The order in which systems must be recovered based on service dependencies (e.g., AD before domain members)

Tools & Systems

  • Veeam Instant Recovery: Boots VMs directly from backup with near-zero RTO, then live-migrates to production
  • Microsoft DSRM: AD-specific recovery mode for restoring domain controllers from backup
  • DSInternals PowerShell Module: Validates AD database integrity and identifies compromised credentials post-recovery
  • Rubrik Instant Recovery: Mounts backup as live VM in seconds for rapid recovery validation
  • ClamAV: Open-source antivirus for scanning backup files before restoration

Common Scenarios

Scenario: Manufacturing Company Full Recovery After LockBit Attack

Context: A manufacturer with 300 servers has 80% of infrastructure encrypted by LockBit. Immutable backups from 48 hours ago are verified clean. Production lines are down, costing $500K/day.

Approach:

  1. Establish recovery VLAN (10.99.0.0/24) isolated from compromised network
  2. Restore 2 domain controllers from immutable backup using Veeam Instant Recovery (2 hours)
  3. Reset krbtgt password twice with 12-hour gap, reset all admin passwords
  4. Validate AD with dcdiag, scan for Golden Ticket indicators with DSInternals
  5. Restore ERP database (SAP) and verify data consistency (4 hours)
  6. Restore MES (Manufacturing Execution System) and SCADA historians (3 hours)
  7. Bring production line controllers online in isolated OT network first
  8. Phased reconnection over 48 hours with continuous EDR monitoring
  9. Total recovery: 72 hours (within 96-hour RTO commitment)

Pitfalls:

  • Rushing to reconnect systems without validating absence of persistence mechanisms, causing re-infection
  • Restoring from the most recent backup without verifying it predates the compromise (attacker may have poisoned recent backups)
  • Not resetting the krbtgt password twice, allowing attackers to maintain Golden Ticket access
  • Restoring systems in the wrong order (application servers before their database dependencies)

Output Format

## Ransomware Recovery Status Report
 
**Incident ID**: [ID]
**Recovery Start**: [Timestamp]
**Current Phase**: [1-4]
**Estimated Completion**: [Timestamp]
 
### Recovery Progress
| Phase | Systems | Status | Started | Completed | RTO Target |
|-------|---------|--------|---------|-----------|------------|
| 1 - Identity | DC01, DC02, DNS | Complete | HH:MM | HH:MM | 4 hours |
| 2 - Critical | ERP, DB01, DB02 | In Progress | HH:MM | -- | 12 hours |
| 3 - Important | FS01, Email, Web | Pending | -- | -- | 24 hours |
| 4 - Remaining | Dev, Archive | Pending | -- | -- | 48 hours |
 
### Validation Checklist
- [ ] AD integrity verified (dcdiag, repadmin)
- [ ] krbtgt password reset (2x with interval)
- [ ] All admin passwords reset
- [ ] Persistence mechanisms scanned
- [ ] EDR deployed and active on recovered systems
- [ ] IOCs loaded in detection rules
- [ ] Canary files deployed
Source materials

References and resources

Everything below is rendered for inspection. Script files are read-only and never run.

References 3

api-reference.md1.7 KB

API Reference: Recovering from Ransomware Attack

Recovery Priority Order

Priority Systems Why First
1 Domain Controllers All auth depends on AD
2 DNS/DHCP Network functionality
3 Authentication (SSO/MFA) User access
4 Email Communication
5 Database Servers Business data
6 Application Servers Business operations
7 File Servers Data access
8 Workstations End user devices

KRBTGT Reset Procedure

Step Command Note
1 Reset-KrbtgtPassword First reset
2 Wait 12 hours Allow replication
3 Reset-KrbtgtPassword Second reset
4 dcdiag /v Validate DC health

Backup Verification Commands

Command Description
veeamcli verify Verify Veeam backup integrity
wbadmin get versions List Windows Server backups
aws s3api head-object Check S3 backup metadata

3-2-1-1-0 Backup Strategy

Component Description
3 copies Production + 2 backups
2 media types Disk + tape/cloud
1 offsite Geographically separate
1 offline Air-gapped or immutable
0 errors Verified with restore tests

Python Libraries

Library Version Purpose
json stdlib Recovery tracking
datetime stdlib Timeline documentation
pathlib stdlib Backup path verification

References

standards.md1.0 KB

Standards & References - Recovering from Ransomware Attack

Frameworks

  • NIST SP 800-61r3: Computer Security Incident Handling Guide (Recover phase)
  • NIST IR 8374: Ransomware Risk Management - Recovery section
  • CISA #StopRansomware Guide: Recovery checklist
  • CIS Controls v8: Control 11 (Data Recovery)
  • NIST CSF 2.0: Recover function (RC.RP, RC.CO)

AD Recovery

MITRE ATT&CK (Recovery Validation)

  • T1053: Scheduled Task/Job (persistence to check)
  • T1543: Create or Modify System Process (persistence to check)
  • T1547: Boot or Logon Autostart Execution (persistence to check)
  • T1558.001: Golden Ticket (must reset krbtgt)
  • T1098: Account Manipulation (check for backdoor accounts)
workflows.md1.4 KB

Workflows - Recovering from Ransomware Attack

Workflow 1: Recovery Decision and Planning

Containment Complete + Forensics Initiated
  |
  v
[Assess backup availability]
  |-- Immutable copies intact? --> Primary recovery path
  |-- Air-gapped copies available? --> Secondary recovery path
  |-- No clean backups? --> Consider decryption key (paid or NoMoreRansom)
  |
  v
[Determine recovery scope]
  |-- Full rebuild vs. selective restore
  |-- Identify minimum viable recovery set
  |
  v
[Map system dependencies]
  |-- AD/DNS -> Database -> Application -> Web
  |
  v
[Estimate recovery timeline per tier]
  |
  v
[Brief executive team on recovery plan and timeline]
  |
  v
[Begin recovery]

Workflow 2: System Recovery Execution

[Establish clean recovery VLAN]
  |
  v
[Phase 1: Identity Recovery]
  |-- Restore DCs from verified backup
  |-- Reset krbtgt (2x)
  |-- Reset all admin passwords
  |-- Validate AD health (dcdiag)
  |
  v
[Phase 2: Critical Systems]
  |-- Restore databases
  |-- Verify data consistency
  |-- Restore core applications
  |-- Test application functionality
  |
  v
[Phase 3: Important Systems]
  |-- Restore in groups of 5-10
  |-- Validate each before proceeding
  |
  v
[Phase 4: Remaining Systems]
  |
  v
[Each system: Scan for persistence -> Patch -> Deploy EDR -> Connect]
  |
  v
[Post-recovery monitoring (7-14 days elevated alert)]

Scripts 2

agent.py6.6 KB
Display-only source. This catalog never executes bundled scripts.
#!/usr/bin/env python3
"""Agent for ransomware attack recovery coordination.

Manages recovery workflow: backup verification, system rebuild
prioritization, credential reset tracking, and post-recovery
validation checklists with timeline documentation.
"""

import json
import sys
from datetime import datetime
from pathlib import Path


RECOVERY_PRIORITY = [
    ("Domain Controllers", "critical", "Rebuild from clean media first"),
    ("DNS/DHCP Servers", "critical", "Required for network functionality"),
    ("Authentication Services", "critical", "SSO, MFA, RADIUS"),
    ("Email Server", "high", "Communication during recovery"),
    ("Database Servers", "high", "Restore from verified clean backup"),
    ("Application Servers", "high", "Business-critical applications"),
    ("File Servers", "medium", "Restore data from backup"),
    ("Workstations", "medium", "Reimage, do not file-level restore"),
    ("Print Servers", "low", "Rebuild after core services"),
]


class RansomwareRecoveryAgent:
    """Coordinates ransomware attack recovery procedures."""

    def __init__(self, case_id, output_dir="./recovery"):
        self.case_id = case_id
        self.output_dir = Path(output_dir)
        self.output_dir.mkdir(parents=True, exist_ok=True)
        self.recovery = {
            "case_id": case_id, "status": "in_progress",
            "timeline": [], "systems": [], "checklists": {},
        }

    def log_event(self, event_type, description):
        self.recovery["timeline"].append({
            "timestamp": datetime.utcnow().isoformat(),
            "type": event_type, "description": description,
        })

    def verify_backup(self, backup_path, backup_type, last_verified=None):
        """Record backup verification status."""
        status = {
            "path": backup_path, "type": backup_type,
            "verified_date": last_verified or datetime.utcnow().isoformat(),
            "integrity": "pending",
        }
        if Path(backup_path).exists() if not backup_path.startswith("s3://") else True:
            status["integrity"] = "accessible"
        self.recovery.setdefault("backups", []).append(status)
        self.log_event("backup_check", f"Verified {backup_type}: {backup_path}")
        return status

    def build_recovery_plan(self, clean_backup_available=True):
        """Generate prioritized system recovery plan."""
        systems = []
        for name, priority, note in RECOVERY_PRIORITY:
            systems.append({
                "system": name, "priority": priority, "note": note,
                "status": "pending", "recovery_method": (
                    "Restore from backup" if clean_backup_available
                    else "Rebuild from scratch"
                ),
            })
        self.recovery["systems"] = systems
        self.log_event("plan_created", f"{len(systems)} systems in recovery plan")
        return systems

    def update_system_status(self, system_name, status, notes=""):
        for sys_entry in self.recovery["systems"]:
            if sys_entry["system"] == system_name:
                sys_entry["status"] = status
                if notes:
                    sys_entry["recovery_notes"] = notes
                self.log_event("system_update",
                               f"{system_name}: {status}")
                return sys_entry
        return None

    def generate_credential_reset_checklist(self):
        """Generate credential reset checklist for post-recovery."""
        checklist = [
            {"item": "Reset KRBTGT password (twice, 12h apart)", "status": "pending",
             "priority": "critical"},
            {"item": "Reset all Domain Admin passwords", "status": "pending",
             "priority": "critical"},
            {"item": "Reset all service account passwords", "status": "pending",
             "priority": "critical"},
            {"item": "Reset all user passwords", "status": "pending",
             "priority": "high"},
            {"item": "Revoke and reissue all certificates", "status": "pending",
             "priority": "high"},
            {"item": "Rotate all API keys and tokens", "status": "pending",
             "priority": "high"},
            {"item": "Reset cloud IAM credentials", "status": "pending",
             "priority": "high"},
            {"item": "Deploy LAPS for local admin passwords", "status": "pending",
             "priority": "medium"},
        ]
        self.recovery["checklists"]["credential_reset"] = checklist
        return checklist

    def generate_hardening_checklist(self):
        """Post-recovery hardening recommendations."""
        checklist = [
            {"item": "Enforce MFA on all remote access", "status": "pending"},
            {"item": "Implement 3-2-1-1-0 backup strategy", "status": "pending"},
            {"item": "Deploy EDR on all endpoints", "status": "pending"},
            {"item": "Enable PowerShell Script Block Logging", "status": "pending"},
            {"item": "Implement network segmentation", "status": "pending"},
            {"item": "Block SMB between workstations", "status": "pending"},
            {"item": "Disable NTLM where possible", "status": "pending"},
            {"item": "Deploy application whitelisting on servers", "status": "pending"},
            {"item": "Implement privileged access workstations", "status": "pending"},
        ]
        self.recovery["checklists"]["post_hardening"] = checklist
        return checklist

    def get_recovery_progress(self):
        """Calculate overall recovery progress."""
        total = len(self.recovery["systems"])
        completed = sum(1 for s in self.recovery["systems"]
                        if s["status"] in ("recovered", "verified"))
        return {
            "total_systems": total,
            "recovered": completed,
            "progress_pct": round(completed / max(total, 1) * 100, 1),
        }

    def generate_report(self):
        self.generate_credential_reset_checklist()
        self.generate_hardening_checklist()
        progress = self.get_recovery_progress()

        report = {
            **self.recovery,
            "progress": progress,
            "report_date": datetime.utcnow().isoformat(),
        }
        report_path = self.output_dir / f"{self.case_id}_recovery.json"
        with open(report_path, "w") as f:
            json.dump(report, f, indent=2, default=str)
        print(json.dumps(report, indent=2, default=str))
        return report


def main():
    case_id = sys.argv[1] if len(sys.argv) > 1 else "RAN-2025-001"
    agent = RansomwareRecoveryAgent(case_id)
    agent.build_recovery_plan(clean_backup_available=True)
    agent.generate_report()


if __name__ == "__main__":
    main()
process.py10.9 KB
Display-only source. This catalog never executes bundled scripts.
#!/usr/bin/env python3
"""
Ransomware Recovery Orchestration and Tracking Tool

Tracks recovery progress across multiple systems and phases:
- Recovery phase tracking with dependency management
- RTO compliance monitoring
- System validation checklists
- Recovery status reporting
"""

import json
import sys
from dataclasses import dataclass, field, asdict
from datetime import datetime, timedelta
from pathlib import Path
from typing import Optional


@dataclass
class RecoverableSystem:
    name: str
    tier: int
    system_type: str  # dc, database, application, fileserver, web, other
    dependencies: list = field(default_factory=list)
    rto_hours: float = 24.0
    status: str = "pending"  # pending, restoring, validating, online, failed
    backup_source: str = ""
    backup_timestamp: Optional[str] = None
    restore_start: Optional[str] = None
    restore_end: Optional[str] = None
    validation_checks: dict = field(default_factory=dict)
    notes: str = ""


@dataclass
class RecoveryPlan:
    incident_id: str
    organization: str
    recovery_start: str
    compromise_date: str
    systems: list = field(default_factory=list)
    phases: dict = field(default_factory=dict)
    overall_status: str = "in_progress"


class RecoveryOrchestrator:
    """Manages and tracks ransomware recovery across all systems."""

    def __init__(self, incident_id: str, org_name: str, compromise_date: str):
        self.plan = RecoveryPlan(
            incident_id=incident_id,
            organization=org_name,
            recovery_start=datetime.now().isoformat(),
            compromise_date=compromise_date,
        )

    def add_system(self, system: RecoverableSystem):
        self.plan.systems.append(system)

    def start_restore(self, system_name: str, backup_source: str):
        for sys in self.plan.systems:
            if sys.name == system_name:
                # Check dependencies are online
                for dep in sys.dependencies:
                    dep_sys = next((s for s in self.plan.systems if s.name == dep), None)
                    if dep_sys and dep_sys.status != "online":
                        raise ValueError(
                            f"Cannot restore {system_name}: dependency {dep} "
                            f"is {dep_sys.status}, must be 'online'"
                        )
                sys.status = "restoring"
                sys.backup_source = backup_source
                sys.restore_start = datetime.now().isoformat()
                return
        raise ValueError(f"System not found: {system_name}")

    def complete_restore(self, system_name: str):
        for sys in self.plan.systems:
            if sys.name == system_name:
                sys.status = "validating"
                sys.restore_end = datetime.now().isoformat()
                return
        raise ValueError(f"System not found: {system_name}")

    def validate_system(self, system_name: str, checks: dict):
        """Record validation check results for a restored system."""
        for sys in self.plan.systems:
            if sys.name == system_name:
                sys.validation_checks = checks
                all_passed = all(checks.values())
                sys.status = "online" if all_passed else "failed"
                return all_passed
        raise ValueError(f"System not found: {system_name}")

    def check_rto_compliance(self) -> list:
        """Check which systems are at risk of exceeding their RTO."""
        violations = []
        recovery_start = datetime.fromisoformat(self.plan.recovery_start)

        for sys in self.plan.systems:
            rto_deadline = recovery_start + timedelta(hours=sys.rto_hours)

            if sys.status in ("pending", "restoring", "validating"):
                if datetime.now() > rto_deadline:
                    violations.append({
                        "system": sys.name,
                        "tier": sys.tier,
                        "rto_hours": sys.rto_hours,
                        "deadline": rto_deadline.isoformat(),
                        "status": sys.status,
                        "exceeded_by_hours": round(
                            (datetime.now() - rto_deadline).total_seconds() / 3600, 1
                        ),
                    })
        return violations

    def get_recovery_progress(self) -> dict:
        """Calculate overall recovery progress."""
        total = len(self.plan.systems)
        if total == 0:
            return {"progress": 0, "by_status": {}, "by_tier": {}}

        by_status = {}
        by_tier = {}
        for sys in self.plan.systems:
            by_status[sys.status] = by_status.get(sys.status, 0) + 1
            tier_key = f"tier_{sys.tier}"
            if tier_key not in by_tier:
                by_tier[tier_key] = {"total": 0, "online": 0}
            by_tier[tier_key]["total"] += 1
            if sys.status == "online":
                by_tier[tier_key]["online"] += 1

        online = by_status.get("online", 0)
        return {
            "progress": round((online / total) * 100, 1),
            "total_systems": total,
            "by_status": by_status,
            "by_tier": by_tier,
        }

    def get_next_recoverable(self) -> list:
        """Get list of systems ready for recovery (dependencies met)."""
        ready = []
        for sys in self.plan.systems:
            if sys.status != "pending":
                continue
            deps_met = all(
                next((s for s in self.plan.systems if s.name == dep), None) is not None
                and next((s for s in self.plan.systems if s.name == dep)).status == "online"
                for dep in sys.dependencies
            )
            if deps_met or not sys.dependencies:
                ready.append(sys)
        return sorted(ready, key=lambda s: s.tier)

    def generate_report(self) -> str:
        lines = []
        lines.append("=" * 70)
        lines.append("RANSOMWARE RECOVERY STATUS REPORT")
        lines.append("=" * 70)
        lines.append(f"Incident: {self.plan.incident_id}")
        lines.append(f"Organization: {self.plan.organization}")
        lines.append(f"Recovery Started: {self.plan.recovery_start}")
        lines.append(f"Compromise Date: {self.plan.compromise_date}")

        progress = self.get_recovery_progress()
        lines.append(f"\nOverall Progress: {progress['progress']}%")
        lines.append(f"Total Systems: {progress['total_systems']}")

        for status, count in progress["by_status"].items():
            lines.append(f"  {status}: {count}")

        lines.append("\nBy Tier:")
        for tier, data in sorted(progress["by_tier"].items()):
            lines.append(f"  {tier}: {data['online']}/{data['total']} online")

        # RTO violations
        violations = self.check_rto_compliance()
        if violations:
            lines.append(f"\nRTO VIOLATIONS ({len(violations)}):")
            for v in violations:
                lines.append(f"  {v['system']} (Tier {v['tier']}): "
                           f"RTO {v['rto_hours']}h exceeded by {v['exceeded_by_hours']}h")

        # System details
        lines.append("\nSystem Recovery Status:")
        lines.append("-" * 50)
        for tier in sorted(set(s.tier for s in self.plan.systems)):
            tier_systems = [s for s in self.plan.systems if s.tier == tier]
            lines.append(f"\n  Tier {tier}:")
            for sys in tier_systems:
                status_icon = {"pending": "[ ]", "restoring": "[~]", "validating": "[?]",
                              "online": "[+]", "failed": "[X]"}.get(sys.status, "[?]")
                lines.append(f"    {status_icon} {sys.name} ({sys.system_type}) - {sys.status}")
                if sys.restore_start and sys.restore_end:
                    start = datetime.fromisoformat(sys.restore_start)
                    end = datetime.fromisoformat(sys.restore_end)
                    duration = (end - start).total_seconds() / 3600
                    lines.append(f"        Restore time: {duration:.1f} hours")
                if sys.validation_checks:
                    for check, passed in sys.validation_checks.items():
                        result = "PASS" if passed else "FAIL"
                        lines.append(f"        {check}: {result}")

        # Next recoverable systems
        ready = self.get_next_recoverable()
        if ready:
            lines.append(f"\nReady for Recovery ({len(ready)}):")
            for sys in ready:
                lines.append(f"  - {sys.name} (Tier {sys.tier}, {sys.system_type})")

        lines.append("\n" + "=" * 70)
        return "\n".join(lines)

    def export_plan(self, output_path: str):
        with open(output_path, "w") as f:
            json.dump(asdict(self.plan), f, indent=2)


def main():
    """Demo recovery orchestration."""
    orch = RecoveryOrchestrator(
        incident_id="INC-2026-0042",
        org_name="Acme Manufacturing",
        compromise_date="2026-02-20T03:00:00",
    )

    # Define systems with dependencies
    orch.add_system(RecoverableSystem(
        name="DC01", tier=1, system_type="dc", rto_hours=4,
    ))
    orch.add_system(RecoverableSystem(
        name="DC02", tier=1, system_type="dc", rto_hours=4,
    ))
    orch.add_system(RecoverableSystem(
        name="DNS01", tier=1, system_type="dns", rto_hours=4,
        dependencies=["DC01"],
    ))
    orch.add_system(RecoverableSystem(
        name="SQL-ERP", tier=1, system_type="database", rto_hours=8,
        dependencies=["DC01", "DNS01"],
    ))
    orch.add_system(RecoverableSystem(
        name="ERP-APP", tier=1, system_type="application", rto_hours=12,
        dependencies=["SQL-ERP", "DC01"],
    ))
    orch.add_system(RecoverableSystem(
        name="Exchange", tier=2, system_type="email", rto_hours=12,
        dependencies=["DC01", "DC02"],
    ))
    orch.add_system(RecoverableSystem(
        name="FileServer01", tier=2, system_type="fileserver", rto_hours=24,
        dependencies=["DC01"],
    ))
    orch.add_system(RecoverableSystem(
        name="WebApp01", tier=2, system_type="web", rto_hours=24,
        dependencies=["SQL-ERP"],
    ))
    orch.add_system(RecoverableSystem(
        name="DevServer", tier=3, system_type="other", rto_hours=48,
        dependencies=["DC01"],
    ))

    # Simulate recovery progress
    orch.start_restore("DC01", "Immutable Veeam Hardened Repo")
    orch.complete_restore("DC01")
    orch.validate_system("DC01", {
        "dcdiag": True, "repadmin": True, "dns_resolution": True,
        "krbtgt_reset": True, "persistence_scan": True,
    })

    orch.start_restore("DC02", "Immutable Veeam Hardened Repo")
    orch.complete_restore("DC02")
    orch.validate_system("DC02", {
        "dcdiag": True, "repadmin": True, "replication": True,
    })

    orch.start_restore("DNS01", "Immutable Veeam Hardened Repo")

    print(orch.generate_report())

    output_path = str(Path(__file__).parent / "recovery_plan.json")
    orch.export_plan(output_path)
    print(f"\nRecovery plan exported to: {output_path}")


if __name__ == "__main__":
    main()
Keep exploring