npx skills add mukul975/Anthropic-Cybersecurity-SkillsMITRE ATT&CK
Overview
The NTFS Master File Table ($MFT) is the central metadata repository for every file and directory on an NTFS volume. Each file is represented by at least one 1024-byte MFT record containing attributes such as $STANDARD_INFORMATION (timestamps, permissions), $FILE_NAME (name, parent directory, timestamps), and $DATA (file content or cluster run pointers). When a file is deleted, its MFT record is marked as inactive (InUse flag cleared) but the metadata remains until the entry is reallocated by a new file. This persistence makes MFT analysis a primary technique for recovering deleted file evidence, reconstructing file system timelines, and detecting anti-forensic activity such as timestomping.
When to Use
- When investigating security incidents that require analyzing mft for deleted file recovery
- When building detection rules or threat hunting queries for this domain
- When SOC analysts need structured procedures for this analysis type
- When validating security monitoring coverage for related attack techniques
Prerequisites
- Forensic disk image (E01, raw/dd, VMDK, or VHDX format)
- MFTECmd (Eric Zimmerman) or analyzeMFT (Python-based)
- FTK Imager, Arsenal Image Mounter, or similar for image mounting
- Timeline Explorer or Excel for CSV analysis
- Python 3.8+ for custom analysis scripts
- Understanding of NTFS file system internals
MFT Structure and Record Layout
MFT Record Header
Each MFT record begins with the signature "FILE" (0x46494C45) and contains:
| Offset | Size | Field |
|---|---|---|
| 0x00 | 4 bytes | Signature ("FILE") |
| 0x04 | 2 bytes | Offset to update sequence |
| 0x06 | 2 bytes | Size of update sequence |
| 0x08 | 8 bytes | $LogFile sequence number |
| 0x10 | 2 bytes | Sequence number |
| 0x12 | 2 bytes | Hard link count |
| 0x14 | 2 bytes | Offset to first attribute |
| 0x16 | 2 bytes | Flags (0x01 = InUse, 0x02 = Directory) |
| 0x18 | 4 bytes | Used size of MFT record |
| 0x1C | 4 bytes | Allocated size of MFT record |
| 0x20 | 8 bytes | Base file record reference |
| 0x28 | 2 bytes | Next attribute ID |
Key MFT Attributes
| Type ID | Name | Description |
|---|---|---|
| 0x10 | $STANDARD_INFORMATION | Timestamps, flags, owner ID, security ID |
| 0x30 | $FILE_NAME | Filename, parent MFT reference, timestamps |
| 0x40 | $OBJECT_ID | Unique GUID for the file |
| 0x50 | $SECURITY_DESCRIPTOR | ACL permissions |
| 0x60 | $VOLUME_NAME | Volume label (volume metadata files only) |
| 0x80 | $DATA | File content (resident if <700 bytes) or cluster run list |
| 0x90 | $INDEX_ROOT | B-tree index root for directories |
| 0xA0 | $INDEX_ALLOCATION | B-tree index entries for large directories |
| 0xB0 | $BITMAP | Allocation bitmap for index or MFT |
Deleted File Recovery Techniques
Technique 1: MFT Record Analysis with MFTECmd
# Extract $MFT from forensic image using KAPE or FTK Imager
# Parse the $MFT with MFTECmd
MFTECmd.exe -f "C:\Evidence\$MFT" --csv C:\Output --csvf mft_full.csv
# Filter for deleted files (InUse = FALSE) in Timeline Explorer
# Look for entries where InUse column is FalseIdentifying Deleted Files in CSV Output:
InUse= False indicates a deleted or reallocated recordParentPathshows original file location before deletionFileSizeshows the original size (may still be recoverable)- Timestamps in
$STANDARD_INFORMATIONand$FILE_NAMEattributes persist
Technique 2: USN Journal ($UsnJrnl:$J) Analysis
The USN Journal records all changes to files on an NTFS volume, including creation, deletion, rename, and data modification events.
# Parse USN Journal with MFTECmd
MFTECmd.exe -f "C:\Evidence\$J" --csv C:\Output --csvf usn_journal.csv
# Key USN reason codes for deletion evidence:
# USN_REASON_FILE_DELETE = 0x00000200
# USN_REASON_CLOSE = 0x80000000
# USN_REASON_RENAME_OLD_NAME = 0x00001000
# USN_REASON_RENAME_NEW_NAME = 0x00002000Technique 3: $LogFile Transaction Analysis
The $LogFile stores NTFS transaction records that can reveal file operations even after the USN Journal has been cycled.
# Parse $LogFile with LogFileParser
LogFileParser.exe -l "C:\Evidence\$LogFile" -o C:\Output
# Look for REDO and UNDO operations indicating file deletion:
# - DeallocateFileRecordSegment
# - DeleteAttribute
# - UpdateResidentValue (clearing InUse flag)Technique 4: MFT Slack Space Analysis
MFT slack space exists between the end of the used portion of an MFT record and the end of the allocated 1024 bytes. This area may contain remnants of previous file records.
import struct
def parse_mft_slack(mft_path: str, output_path: str):
"""Extract and analyze MFT slack space for deleted file remnants."""
with open(mft_path, "rb") as f:
record_size = 1024
record_num = 0
slack_findings = []
while True:
record = f.read(record_size)
if len(record) < record_size:
break
# Verify FILE signature
if record[:4] != b"FILE":
record_num += 1
continue
# Get used size from offset 0x18
used_size = struct.unpack("<I", record[0x18:0x1C])[0]
if used_size < record_size:
slack = record[used_size:]
# Check if slack contains readable strings or attribute headers
if any(c > 0x20 and c < 0x7F for c in slack[:50]):
slack_findings.append({
"record": record_num,
"used_size": used_size,
"slack_size": record_size - used_size,
"slack_preview": slack[:100].hex()
})
record_num += 1
return slack_findingsCorrelation with Supporting Artifacts
Cross-Reference MFT with $Recycle.Bin
# Parse Recycle Bin with RBCmd
RBCmd.exe -d "C:\Evidence\$Recycle.Bin" --csv C:\Output --csvf recycle_bin.csv
# Correlate: $I files contain original path and deletion timestamp
# Match MFT entry numbers from $R files back to original MFT recordsCross-Reference MFT with Volume Shadow Copies
# List volume shadow copies
vssadmin list shadows
# Mount shadow copies and extract $MFT from each
# Compare MFT records across shadow copies to track file changes over timeForensic Value
- Deleted file metadata recovery: Original filename, path, size, and timestamps
- Timeline reconstruction: File creation, modification, access, and deletion events
- Timestomping detection: Comparing $SI vs $FN timestamps
- Data carving guidance: MFT cluster runs point to file content on disk
- Anti-forensic detection: Identifying wiped or manipulated MFT records
References
- NTFS MFT Advanced Forensic Analysis: https://www.deaddisk.com/posts/ntfs-mft-advanced-forensic-analysis-guide/
- MFT Slack Space Forensic Value: https://www.sygnia.co/blog/the-forensic-value-of-mft-slack-space/
- MFTECmd Documentation: https://ericzimmerman.github.io/
- SANS FOR500: Windows Forensic Analysis
Example Output
$ MFTECmd.exe -f "C:\Evidence\$MFT" --csv /analysis/mft_output
MFTECmd v1.2.2 - MFT Parser
==============================
Input: C:\Evidence\$MFT (Size: 384 MB)
Total MFT Entries: 395,264
Parsing MFT entries... Done (12.4 seconds)
--- Deleted File Recovery Summary ---
Total Entries: 395,264
Active Files: 245,832
Deleted Files: 149,432
Recoverable: 87,234 (resident data or clusters not reallocated)
Partially Recoverable: 31,456 (some clusters overwritten)
Unrecoverable: 30,742 (all clusters reallocated)
--- Recently Deleted Files (Incident Window: 2024-01-15 to 2024-01-18) ---
MFT Entry | Filename | Path | Size | Deleted (UTC) | Recoverable
----------|-----------------------------------|------------------------------------|-----------|-----------------------|------------
148923 | exfil_tool.exe | C:\ProgramData\Updates\ | 1,258,496 | 2024-01-17 02:45:12 | YES
148924 | exfil_tool.log | C:\ProgramData\Updates\ | 45,312 | 2024-01-17 02:45:14 | YES
149001 | passwords.txt | C:\Users\jsmith\Desktop\ | 2,048 | 2024-01-17 02:50:33 | YES
149150 | scan_results.csv | C:\Users\jsmith\AppData\Local\Temp | 892,416 | 2024-01-17 03:00:01 | PARTIAL
149200 | mimikatz.exe | C:\Windows\Temp\ | 1,250,816 | 2024-01-18 01:15:22 | YES
149201 | sekurlsa.log | C:\Windows\Temp\ | 32,768 | 2024-01-18 01:15:25 | YES
149302 | .bash_history | C:\Users\jsmith\ | 4,096 | 2024-01-18 03:00:00 | NO
149400 | ClearEventLogs.ps1 | C:\Windows\Temp\ | 1,536 | 2024-01-18 03:01:12 | YES
--- $STANDARD_INFORMATION vs $FILE_NAME Timestamp Analysis (Timestomping Detection) ---
MFT Entry | Filename | $SI Created | $FN Created | Delta | Verdict
----------|---------------------|----------------------|----------------------|-----------|----------
148923 | exfil_tool.exe | 2023-06-15 10:00:00 | 2024-01-15 14:34:02 | -214 days | TIMESTOMPED
149200 | mimikatz.exe | 2022-01-01 00:00:00 | 2024-01-16 02:30:15 | -745 days | TIMESTOMPED
Recovered files exported to: /analysis/mft_output/recovered/
Full CSV report: /analysis/mft_output/mft_analysis.csv (395,264 rows)
Timeline CSV: /analysis/mft_output/mft_timeline.csvReferences and resources
Everything below is rendered for inspection. Script files are read-only and never run.
References 3
api-reference.md1.6 KB
API Reference: NTFS MFT Analysis
MFT Entry Structure (1024 bytes)
| Offset | Size | Field |
|---|---|---|
| 0 | 4 | Signature ("FILE") |
| 18 | 2 | Sequence number |
| 20 | 2 | First attribute offset |
| 22 | 2 | Flags (0x01=in use, 0x02=directory) |
MFT Attribute Types
| Type ID | Name | Description |
|---|---|---|
| 0x10 | $STANDARD_INFORMATION | Timestamps, flags, owner |
| 0x20 | $ATTRIBUTE_LIST | List of attributes in other entries |
| 0x30 | $FILE_NAME | Filename and parent reference |
| 0x40 | $OBJECT_ID | Unique object identifier |
| 0x50 | $SECURITY_DESCRIPTOR | ACL and ownership |
| 0x60 | $VOLUME_NAME | Volume label |
| 0x80 | $DATA | File content (resident or non-resident) |
| 0x90 | $INDEX_ROOT | Directory index root |
| 0xA0 | $INDEX_ALLOCATION | Directory index entries |
| 0xB0 | $BITMAP | Bitmap for index allocation |
$STANDARD_INFORMATION Timestamps
| Offset | Size | Field |
|---|---|---|
| 0 | 8 | Creation time (FILETIME) |
| 8 | 8 | Modification time |
| 16 | 8 | MFT modification time |
| 24 | 8 | Access time |
$FILE_NAME Structure
| Offset | Size | Field |
|---|---|---|
| 0 | 8 | Parent directory reference |
| 64 | 1 | Filename length (chars) |
| 65 | 1 | Namespace (0=POSIX, 1=Win32, 2=DOS) |
| 66 | var | Filename (UTF-16LE) |
FILETIME Conversion
FILETIME_EPOCH = datetime(1601, 1, 1)
dt = FILETIME_EPOCH + timedelta(microseconds=filetime // 10)Tools
# Extract MFT with FTK Imager or raw copy
icat /dev/sda1 0 > $MFT
# analyzeMFT
analyzeMFT.py -f $MFT -o mft.csv
# MFTECmd (Eric Zimmerman)
MFTECmd.exe -f $MFT --csv output/standards.md1.1 KB
Standards and References - MFT Deleted File Recovery
Standards
- NIST SP 800-86: Guide to Integrating Forensic Techniques into Incident Response
- ISO/IEC 27037: Guidelines for identification, collection, acquisition and preservation of digital evidence
- SWGDE Best Practices for Computer Forensics
Key Technical References
- NTFS Documentation (Microsoft): File system internals and MFT structure
- MFTECmd by Eric Zimmerman: Primary parsing tool for $MFT, $J, $LogFile, $Boot
- analyzeMFT (Python): Open-source MFT parser for cross-platform analysis
- ntfstool (GitHub): Forensics tool for NTFS parsing, MFT, BitLocker, deleted files
MITRE ATT&CK Mappings
- T1070.004 - Indicator Removal: File Deletion
- T1070.006 - Indicator Removal: Timestomping
- T1485 - Data Destruction
- T1561 - Disk Wipe
NTFS Specifications
- MFT Record Size: 1024 bytes (default)
- MFT Entry 0: $MFT (self-reference)
- MFT Entry 1: $MFTMirr (mirror of first 4 entries)
- MFT Entry 2: $LogFile (transaction log)
- MFT Entry 5: Root directory
- MFT Entry 6: $Bitmap (cluster allocation)
- MFT Entry 8: $BadClus (bad cluster list)
- MFT Entry 11: $Extend (extended metadata)
workflows.md1.0 KB
Workflows - MFT Deleted File Recovery
Workflow 1: Basic Deleted File Discovery
Extract $MFT from forensic image
|
Parse with MFTECmd to CSV
|
Filter for InUse = False (deleted records)
|
Analyze ParentPath, FileName, FileSize
|
Cross-reference with USN Journal for deletion timestamps
|
Document findings with original paths and timestampsWorkflow 2: MFT Slack Space Recovery
Extract raw $MFT binary
|
Parse each 1024-byte record
|
Compare used_size vs allocated_size (1024)
|
Extract slack bytes between used and allocated
|
Search for attribute headers (0x10, 0x30, 0x80)
|
Reconstruct partial file metadata from slack dataWorkflow 3: Timeline Reconstruction
Parse $MFT for all timestamps ($SI and $FN)
|
Parse $J (USN Journal) for change records
|
Parse $LogFile for transaction records
|
Merge into unified timeline
|
Identify file creation, modification, deletion sequences
|
Flag timestomping indicators ($SI Created < $FN Created)Scripts 2
agent.py5.9 KB
#!/usr/bin/env python3
"""MFT Deleted File Recovery Agent - Parses NTFS Master File Table for deleted file artifacts."""
import json
import struct
import os
import logging
import argparse
from datetime import datetime, timedelta
logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s")
logger = logging.getLogger(__name__)
MFT_ENTRY_SIZE = 1024
FILETIME_EPOCH = datetime(1601, 1, 1)
def filetime_to_dt(ft):
"""Convert FILETIME to datetime."""
if ft == 0:
return None
try:
return FILETIME_EPOCH + timedelta(microseconds=ft // 10)
except (OverflowError, OSError):
return None
def parse_mft_entry(data, offset=0):
"""Parse a single MFT entry."""
if len(data) < offset + 48:
return None
signature = data[offset:offset + 4]
if signature != b"FILE":
return None
flags = struct.unpack_from("<H", data, offset + 22)[0]
seq_number = struct.unpack_from("<H", data, offset + 18)[0]
first_attr_offset = struct.unpack_from("<H", data, offset + 20)[0]
entry = {
"flags": flags,
"in_use": bool(flags & 0x01),
"is_directory": bool(flags & 0x02),
"sequence_number": seq_number,
"attributes": [],
}
attr_offset = offset + first_attr_offset
while attr_offset + 4 <= len(data):
attr_type = struct.unpack_from("<I", data, attr_offset)[0]
if attr_type == 0xFFFFFFFF:
break
attr_length = struct.unpack_from("<I", data, attr_offset + 4)[0]
if attr_length == 0 or attr_offset + attr_length > len(data):
break
if attr_type == 0x10: # $STANDARD_INFORMATION
if attr_offset + 24 + 32 <= len(data):
si_offset = attr_offset + struct.unpack_from("<H", data, attr_offset + 20)[0]
if si_offset + 32 <= len(data):
entry["created"] = str(filetime_to_dt(struct.unpack_from("<Q", data, si_offset)[0]))
entry["modified"] = str(filetime_to_dt(struct.unpack_from("<Q", data, si_offset + 8)[0]))
entry["mft_modified"] = str(filetime_to_dt(struct.unpack_from("<Q", data, si_offset + 16)[0]))
entry["accessed"] = str(filetime_to_dt(struct.unpack_from("<Q", data, si_offset + 24)[0]))
elif attr_type == 0x30: # $FILE_NAME
non_res = struct.unpack_from("<B", data, attr_offset + 8)[0]
if non_res == 0:
fn_offset = attr_offset + struct.unpack_from("<H", data, attr_offset + 20)[0]
if fn_offset + 66 <= len(data):
parent_ref = struct.unpack_from("<Q", data, fn_offset)[0] & 0xFFFFFFFFFFFF
name_len = data[fn_offset + 64] if fn_offset + 64 < len(data) else 0
name_ns = data[fn_offset + 65] if fn_offset + 65 < len(data) else 0
if fn_offset + 66 + name_len * 2 <= len(data):
filename = data[fn_offset + 66:fn_offset + 66 + name_len * 2].decode("utf-16-le", errors="ignore")
entry["filename"] = filename
entry["parent_ref"] = parent_ref
entry["name_type"] = {0: "POSIX", 1: "Win32", 2: "DOS", 3: "Win32+DOS"}.get(name_ns, "Unknown")
attr_offset += attr_length
return entry
def parse_mft_file(mft_path):
"""Parse an extracted MFT file."""
entries = []
with open(mft_path, "rb") as f:
data = f.read()
total_entries = len(data) // MFT_ENTRY_SIZE
for i in range(total_entries):
offset = i * MFT_ENTRY_SIZE
entry = parse_mft_entry(data, offset)
if entry:
entry["record_number"] = i
entries.append(entry)
logger.info("Parsed %d MFT entries (%d total records)", len(entries), total_entries)
return entries
def find_deleted_files(entries):
"""Find deleted file entries in MFT."""
deleted = [e for e in entries if not e["in_use"] and e.get("filename")]
logger.info("Found %d deleted file entries", len(deleted))
return deleted
def analyze_deleted_files(deleted):
"""Analyze deleted files for forensic significance."""
findings = []
suspicious_extensions = {".exe", ".dll", ".ps1", ".bat", ".cmd", ".vbs", ".js", ".hta", ".scr"}
for entry in deleted:
fname = entry.get("filename", "").lower()
ext = os.path.splitext(fname)[1]
if ext in suspicious_extensions:
findings.append({
"record": entry["record_number"],
"filename": entry.get("filename"),
"type": "Deleted executable/script",
"severity": "high",
"modified": entry.get("modified"),
})
return findings
def generate_report(entries, deleted, findings):
"""Generate MFT analysis report."""
report = {
"timestamp": datetime.utcnow().isoformat(),
"total_entries": len(entries),
"active_entries": len([e for e in entries if e["in_use"]]),
"deleted_entries": len(deleted),
"suspicious_deleted": len(findings),
"findings": findings[:100],
"deleted_files": [{"record": d["record_number"], "filename": d.get("filename"), "modified": d.get("modified")} for d in deleted[:200]],
}
print(f"MFT REPORT: {len(entries)} entries, {len(deleted)} deleted, {len(findings)} suspicious")
return report
def main():
parser = argparse.ArgumentParser(description="MFT Deleted File Recovery Agent")
parser.add_argument("--mft-file", required=True, help="Path to extracted $MFT file")
parser.add_argument("--output", default="mft_report.json")
args = parser.parse_args()
entries = parse_mft_file(args.mft_file)
deleted = find_deleted_files(entries)
findings = analyze_deleted_files(deleted)
report = generate_report(entries, deleted, findings)
with open(args.output, "w") as f:
json.dump(report, f, indent=2, default=str)
logger.info("Report saved to %s", args.output)
if __name__ == "__main__":
main()
process.py4.4 KB
#!/usr/bin/env python3
"""
MFT Deleted File Recovery Analyzer
Parses MFT CSV output from MFTECmd to identify deleted files,
detect timestomping, and generate recovery reports.
"""
import csv
import json
import sys
import os
from datetime import datetime
from collections import defaultdict
class MFTDeletedFileAnalyzer:
"""Analyze MFTECmd CSV output for deleted file recovery."""
def __init__(self, mft_csv_path: str, output_dir: str):
self.mft_csv_path = mft_csv_path
self.output_dir = output_dir
os.makedirs(output_dir, exist_ok=True)
self.deleted_files = []
self.timestomped_files = []
self.all_records = []
def parse_csv(self):
"""Parse MFTECmd CSV output."""
with open(self.mft_csv_path, "r", encoding="utf-8-sig") as f:
reader = csv.DictReader(f)
for row in reader:
self.all_records.append(row)
if row.get("InUse", "").lower() == "false":
self.deleted_files.append(row)
def detect_timestomping(self):
"""Identify files with timestomping indicators."""
for row in self.all_records:
si_created = row.get("Created0x10", "")
fn_created = row.get("Created0x30", "")
if si_created and fn_created and si_created != fn_created:
try:
si_dt = datetime.fromisoformat(si_created.replace("Z", "+00:00"))
fn_dt = datetime.fromisoformat(fn_created.replace("Z", "+00:00"))
if si_dt < fn_dt:
self.timestomped_files.append({
"entry_number": row.get("EntryNumber", ""),
"filename": row.get("FileName", ""),
"parent_path": row.get("ParentPath", ""),
"si_created": si_created,
"fn_created": fn_created,
"delta_seconds": (fn_dt - si_dt).total_seconds()
})
except (ValueError, TypeError):
continue
def analyze_deleted_by_extension(self) -> dict:
"""Categorize deleted files by extension."""
by_ext = defaultdict(list)
for record in self.deleted_files:
ext = record.get("Extension", "NO_EXT").upper()
by_ext[ext].append({
"filename": record.get("FileName", ""),
"parent_path": record.get("ParentPath", ""),
"file_size": record.get("FileSize", ""),
"created": record.get("Created0x10", ""),
"modified": record.get("LastModified0x10", "")
})
return dict(by_ext)
def generate_report(self) -> str:
"""Generate comprehensive analysis report."""
self.parse_csv()
self.detect_timestomping()
ext_analysis = self.analyze_deleted_by_extension()
report = {
"analysis_timestamp": datetime.now().isoformat(),
"source_file": self.mft_csv_path,
"total_records": len(self.all_records),
"deleted_records": len(self.deleted_files),
"timestomped_records": len(self.timestomped_files),
"deleted_by_extension": {k: len(v) for k, v in ext_analysis.items()},
"timestomping_details": self.timestomped_files[:50],
"notable_deleted_files": [
{
"filename": r.get("FileName", ""),
"parent_path": r.get("ParentPath", ""),
"file_size": r.get("FileSize", ""),
"entry_number": r.get("EntryNumber", "")
}
for r in self.deleted_files[:100]
]
}
report_path = os.path.join(self.output_dir, "mft_deleted_analysis.json")
with open(report_path, "w") as f:
json.dump(report, f, indent=2)
print(f"[*] Total MFT records: {report['total_records']}")
print(f"[*] Deleted records: {report['deleted_records']}")
print(f"[*] Timestomped records: {report['timestomped_records']}")
print(f"[*] Report saved to: {report_path}")
return report_path
def main():
if len(sys.argv) < 3:
print("Usage: python process.py <mft_csv_path> <output_dir>")
sys.exit(1)
analyzer = MFTDeletedFileAnalyzer(sys.argv[1], sys.argv[2])
analyzer.generate_report()
if __name__ == "__main__":
main()