npx skills add mukul975/Anthropic-Cybersecurity-SkillsAuthorized-use-only notice: This skill includes routines that craft poisoned samples and backdoor triggers for defensive validation. Generate and use poisoned data and backdoored models only in isolated test environments you control. Never deploy a backdoored model or distribute poisoned datasets.
Overview
Data poisoning and model backdooring attack the integrity of an ML system at training time rather than at inference. In data poisoning (MITRE ATLAS AML.T0020 Poison Training Data), an adversary injects manipulated samples into the training, fine-tuning, or RAG corpus so the resulting model misbehaves — degraded accuracy, targeted misclassification, or an attacker-chosen bias. In model backdooring (MITRE ATLAS AML.T0018 Backdoor ML Model), the model behaves normally on clean inputs but produces an attacker-chosen output whenever a hidden trigger (a pixel patch, a rare token, a phrase) is present. Both are amplified by ML supply-chain compromise (AML.T0010): poisoned public datasets, trojaned pre-trained weights downloaded from a hub, or a malicious model serialization. This is OWASP LLM04:2025 Data and Model Poisoning.
Detection spans the pipeline. On the data side: provenance and integrity checks, statistical outlier and label-flip detection, and de-duplication of suspiciously near-identical samples. On the model side: activation-clustering and spectral-signature analysis (which exploit the fact that poisoned samples activate the network differently than clean ones) and trigger reconstruction. On the supply-chain side: verifying weights hashes/signatures and refusing unsafe serialization formats (pickle-based .bin/.pt) in favor of safetensors. This skill implements all three using IBM's Adversarial Robustness Toolbox (ART), Cleanlab for label-quality issues, and integrity tooling.
When to Use
- Before training/fine-tuning on third-party or user-contributed data.
- Before deploying a model built on a downloaded pre-trained checkpoint.
- During an ML supply-chain security review.
- When investigating anomalous model behavior tied to specific inputs (possible backdoor trigger).
- As a CI/CD gate that scans datasets and model artifacts before they enter the pipeline.
Prerequisites
- Python 3.10+ and a virtual environment.
- Install the tooling:
python -m venv .venv && source .venv/bin/activate
# IBM Adversarial Robustness Toolbox — poisoning detection defenses
pip install adversarial-robustness-toolbox
# Cleanlab — label/data quality issue detection
pip install cleanlab
# Modeling + safe serialization + hashing
pip install numpy scikit-learn safetensors
# (Choose one framework backend ART can wrap)
pip install tensorflow # or: pip install torchObjectives
- Verify dataset and model-weight provenance and integrity (hashes/signatures, safe formats).
- Detect label-quality issues and outliers in training data with Cleanlab.
- Detect poisoned samples in a trained model using ART activation clustering.
- Confirm findings with ART spectral-signature analysis.
- Probe a suspect model for backdoor triggers and quantify trigger-induced misclassification.
- Produce a poisoning-assessment report mapped to ATLAS AML.T0020 / AML.T0018.
MITRE ATT&CK Mapping
| ID | Official Name | Relevance |
|---|---|---|
| AML.T0020 | Poison Training Data | Injection of manipulated samples into the training corpus |
| AML.T0018 | Backdoor ML Model | Trigger-activated hidden behavior in the trained model |
| AML.T0010 | ML Supply Chain Compromise | Poisoned public datasets / trojaned downloaded weights |
| AML.T0024 | Exfiltration via ML Inference API | Some poisoning aims to leak data via the model's responses |
Workflow
1. Verify data and model provenance/integrity
Refuse artifacts whose hash/signature you cannot verify, and prefer safetensors over pickle-based formats (pickle can execute code on load).
# Verify a downloaded checkpoint against a published SHA-256
sha256sum model.safetensors
# compare to the hub-published digest
# Flag unsafe pickle-based weights in a directory
find ./models -type f \( -name "*.bin" -o -name "*.pt" -o -name "*.pkl" -o -name "*.ckpt" \)# safe_load.py — load weights without executing pickle
from safetensors.numpy import load_file
weights = load_file("model.safetensors") # no arbitrary code execution2. Detect label/data-quality issues with Cleanlab
Cleanlab finds mislabeled, outlier, and near-duplicate samples — common signatures of label-flip poisoning.
# cleanlab_scan.py
import numpy as np
from cleanlab.filter import find_label_issues
# pred_probs: out-of-sample predicted probabilities (n_samples x n_classes)
# labels: given integer labels (n_samples,)
def scan(labels: np.ndarray, pred_probs: np.ndarray):
issues = find_label_issues(
labels=labels, pred_probs=pred_probs,
return_indices_ranked_by="self_confidence",
)
print(f"[*] {len(issues)} suspected label issues (potential poisoning)")
return issues3. Detect poisoned samples via ART activation clustering
ActivationDefence clusters per-class activations; a class whose activations split into two distinct clusters indicates injected (poisoned) samples.
# activation_defence.py
import numpy as np
from art.estimators.classification import KerasClassifier
from art.defences.detector.poison import ActivationDefence
def detect(model, x_train, y_train):
classifier = KerasClassifier(model=model) # wrap your trained model
defence = ActivationDefence(classifier, x_train, y_train)
report, is_clean_lst = defence.detect_poison(
nb_clusters=2, nb_dims=10, reduce="PCA"
)
# is_clean_lst[i] == 0 marks a suspected poisoned sample
poisoned_idx = np.where(np.array(is_clean_lst) == 0)[0]
print(f"[*] activation clustering flagged {len(poisoned_idx)} samples")
return poisoned_idx, report4. Confirm with ART spectral signatures
Spectral signatures use the covariance spectrum of feature representations to surface poisoned samples — a strong second signal.
# spectral.py
import numpy as np
from art.estimators.classification import KerasClassifier
from art.defences.detector.poison import SpectralSignatureDefense
def detect(model, x_train, y_train, nb_classes):
classifier = KerasClassifier(model=model)
defence = SpectralSignatureDefense(
classifier, x_train, y_train,
expected_pp_poison=0.05, batch_size=128, eps_multiplier=1.5,
)
report, is_clean_lst = defence.detect_poison()
poisoned_idx = np.where(np.array(is_clean_lst) == 0)[0]
print(f"[*] spectral signatures flagged {len(poisoned_idx)} samples")
return poisoned_idx, report5. Probe the model for backdoor triggers
Test whether a candidate trigger flips predictions to an attacker target class far above the clean baseline.
# trigger_probe.py
import numpy as np
def test_trigger(model, x_clean, target_class, apply_trigger):
"""apply_trigger(x) stamps a candidate trigger (e.g. a corner pixel patch)."""
clean_preds = model.predict(x_clean).argmax(axis=1)
x_trig = np.stack([apply_trigger(x.copy()) for x in x_clean])
trig_preds = model.predict(x_trig).argmax(axis=1)
asr = float(np.mean(trig_preds == target_class)) # attack success rate
base = float(np.mean(clean_preds == target_class))
print(f"[*] target-class rate clean={base:.3f} triggered={asr:.3f}")
return {"baseline": base, "trigger_success_rate": asr,
"backdoor_suspected": asr - base > 0.5}6. Quarantine, retrain, and report
Remove flagged samples (intersection of Cleanlab + ART signals is highest-confidence), retrain on the cleaned set, and re-test for the trigger. Document: artifact provenance, samples flagged by each method, trigger ASR before/after, and ATLAS mapping. Recommend dataset provenance controls, signed weights (safetensors + sigstore/cosign), and ongoing pipeline scanning.
Tools and Resources
| Tool | Purpose | Source |
|---|---|---|
| Adversarial Robustness Toolbox | Activation clustering & spectral-signature poisoning defenses | https://github.com/Trusted-AI/adversarial-robustness-toolbox |
| Cleanlab | Label/data-quality issue detection | https://github.com/cleanlab/cleanlab |
| safetensors | Safe (non-pickle) weight serialization | https://github.com/huggingface/safetensors |
| OWASP LLM04:2025 | Data and Model Poisoning reference | https://genai.owasp.org/llmrisk/llm042025-data-and-model-poisoning/ |
| MITRE ATLAS | AI threat technique taxonomy | https://atlas.mitre.org/ |
Detection Method Reference
| Layer | Method | Tool | Signal |
|---|---|---|---|
| Supply chain | Hash/signature + safe format | sha256/safetensors | Tampered or unsafe artifact |
| Data | Label issues / outliers | Cleanlab | Mislabeled / injected samples |
| Model | Activation clustering | ART ActivationDefence | Per-class activation split |
| Model | Spectral signatures | ART SpectralSignatureDefense | Outlier covariance spectrum |
| Model | Trigger probing | custom | High trigger attack-success-rate |
Validation Criteria
- Dataset and weight provenance/integrity verified (hashes, safe format)
- Unsafe pickle-based artifacts identified and avoided
- Cleanlab label-issue scan run and suspicious samples listed
- ART activation clustering executed with flagged sample indices
- ART spectral-signature analysis run as confirmation
- Backdoor trigger probe quantifies attack-success-rate vs. baseline
- Highest-confidence poisoned samples quarantined (multi-method overlap)
- Model retrained on cleaned data and re-tested for the trigger
- Findings mapped to MITRE ATLAS AML.T0020 / AML.T0018 and OWASP LLM04:2025
- Report delivered with remediation (provenance, signed weights, pipeline scanning)
References and resources
Everything below is rendered for inspection. Script files are read-only and never run.
References 2
api-reference.md2.2 KB
API Reference — Data and Model Poisoning Detection
Adversarial Robustness Toolbox (ART)
Install: pip install adversarial-robustness-toolbox
| API | Description |
|---|---|
from art.estimators.classification import KerasClassifier |
Wrap a Keras model for ART (also PyTorchClassifier, TensorFlowV2Classifier) |
from art.defences.detector.poison import ActivationDefence |
Activation-clustering poisoning detector (Chen et al., 2018) |
ActivationDefence(classifier, x_train, y_train) |
Construct the defense |
defence.detect_poison(nb_clusters=2, nb_dims=10, reduce="PCA") |
Returns (report, is_clean_lst); is_clean_lst[i]==0 => poisoned |
from art.defences.detector.poison import SpectralSignatureDefense |
Spectral-signature poisoning detector |
SpectralSignatureDefense(classifier, x, y, expected_pp_poison=0.05, batch_size=128, eps_multiplier=1.5) |
Construct |
defence.detect_poison() |
Returns (report, is_clean_lst) |
Cleanlab
Install: pip install cleanlab
| API | Description |
|---|---|
from cleanlab.filter import find_label_issues |
Find mislabeled samples |
find_label_issues(labels, pred_probs, return_indices_ranked_by="self_confidence") |
Ranked indices of label issues |
from cleanlab.outlier import OutOfDistribution |
Outlier / OOD detection |
from cleanlab import Datalab |
End-to-end data audit (label, outlier, near-duplicate) |
safetensors (safe serialization)
Install: pip install safetensors
| API | Description |
|---|---|
from safetensors.numpy import load_file |
Load weights without executing pickle |
from safetensors.torch import load_file |
PyTorch variant |
Integrity commands
| Command | Purpose |
|---|---|
sha256sum model.safetensors |
Compute weight digest to compare to published value |
find ./models -name "*.pt" -o -name "*.bin" -o -name "*.pkl" |
Locate unsafe pickle-based artifacts |
External References
- ART defenses docs: https://adversarial-robustness-toolbox.readthedocs.io/en/latest/modules/defences/detector_poisoning.html
- Cleanlab docs: https://docs.cleanlab.ai/
- safetensors: https://github.com/huggingface/safetensors
standards.md1.6 KB
Standards and References — Detecting Data and Model Poisoning
MITRE ATLAS References
| Technique ID | Name | Tactic | Rationale |
|---|---|---|---|
| AML.T0020 | Poison Training Data | ML Attack Staging | Injection of manipulated samples into the training corpus |
| AML.T0018 | Backdoor ML Model | Persistence | Trigger-activated hidden behavior in the trained model |
| AML.T0010 | ML Supply Chain Compromise | Initial Access | Poisoned public datasets / trojaned downloaded weights |
| AML.T0024 | Exfiltration via ML Inference API | Exfiltration | Some poisoning leaks data via model responses |
NIST AI RMF References
| ID | Name | Rationale |
|---|---|---|
| MEASURE-2.7 | AI system security and resilience are evaluated and documented | Poisoning detection measures the integrity/resilience of the ML pipeline |
OWASP Top 10 for LLM Applications (2025)
| ID | Name | Rationale |
|---|---|---|
| LLM04:2025 | Data and Model Poisoning | Primary risk this skill detects |
| LLM03:2025 | Supply Chain | Trojaned weights/datasets entry path |
Official Resources
- Adversarial Robustness Toolbox: https://github.com/Trusted-AI/adversarial-robustness-toolbox
- ART poisoning defenses docs: https://adversarial-robustness-toolbox.readthedocs.io/en/latest/modules/defences/detector_poisoning.html
- Cleanlab: https://github.com/cleanlab/cleanlab
- safetensors: https://github.com/huggingface/safetensors
- OWASP LLM04:2025: https://genai.owasp.org/llmrisk/llm042025-data-and-model-poisoning/
- MITRE ATLAS: https://atlas.mitre.org/
- NIST AI RMF: https://www.nist.gov/itl/ai-risk-management-framework
Scripts 1
agent.py4.9 KB
#!/usr/bin/env python3
# For authorized defensive ML-security use in isolated environments only.
"""Data and model poisoning detection agent.
Modes:
integrity -- verify model-weight digests and flag unsafe pickle-based artifacts.
labels -- run Cleanlab label-issue detection from saved labels + pred_probs (.npy).
activations -- run ART activation-clustering poisoning detection on a saved model.
Examples:
python agent.py integrity --model-dir ./models --expected-sha256 ABC...
python agent.py labels --labels labels.npy --pred-probs probs.npy
python agent.py activations --model model.h5 --x x_train.npy --y y_train.npy
"""
import argparse
import glob
import hashlib
import json
import os
import sys
from datetime import datetime, timezone
UNSAFE_EXT = (".pt", ".bin", ".pkl", ".pickle", ".ckpt", ".pth")
def sha256_file(path, chunk=1 << 20):
h = hashlib.sha256()
with open(path, "rb") as fh:
for block in iter(lambda: fh.read(chunk), b""):
h.update(block)
return h.hexdigest()
def run_integrity(args):
report = {"ts": datetime.now(timezone.utc).isoformat(),
"atlas": "AML.T0010", "unsafe_artifacts": [], "digests": {}}
for path in glob.glob(os.path.join(args.model_dir, "**", "*"), recursive=True):
if not os.path.isfile(path):
continue
if path.lower().endswith(UNSAFE_EXT):
report["unsafe_artifacts"].append(path)
if path.lower().endswith(".safetensors"):
report["digests"][path] = sha256_file(path)
if args.expected_sha256:
match = any(d.lower() == args.expected_sha256.lower()
for d in report["digests"].values())
report["expected_digest_match"] = match
if not match:
report["warning"] = "No .safetensors matched the expected digest"
if report["unsafe_artifacts"]:
report["warning_unsafe"] = (
"Pickle-based artifacts can execute code on load; prefer safetensors")
print(json.dumps(report, indent=2))
return report
def run_labels(args):
try:
import numpy as np
from cleanlab.filter import find_label_issues
except ImportError:
print("Install: pip install cleanlab numpy", file=sys.stderr)
sys.exit(1)
labels = np.load(args.labels)
pred_probs = np.load(args.pred_probs)
issues = find_label_issues(labels=labels, pred_probs=pred_probs,
return_indices_ranked_by="self_confidence")
report = {"ts": datetime.now(timezone.utc).isoformat(), "atlas": "AML.T0020",
"n_samples": int(len(labels)), "n_label_issues": int(len(issues)),
"top_suspect_indices": [int(i) for i in issues[:50]]}
print(json.dumps(report, indent=2))
return report
def run_activations(args):
try:
import numpy as np
from tensorflow.keras.models import load_model
from art.estimators.classification import KerasClassifier
from art.defences.detector.poison import ActivationDefence
except ImportError as exc:
print(f"Install: pip install adversarial-robustness-toolbox tensorflow numpy "
f"({exc})", file=sys.stderr)
sys.exit(1)
model = load_model(args.model)
x_train = np.load(args.x)
y_train = np.load(args.y)
classifier = KerasClassifier(model=model)
defence = ActivationDefence(classifier, x_train, y_train)
_, is_clean_lst = defence.detect_poison(nb_clusters=2, nb_dims=10, reduce="PCA")
poisoned = [int(i) for i, c in enumerate(is_clean_lst) if c == 0]
report = {"ts": datetime.now(timezone.utc).isoformat(), "atlas": "AML.T0018",
"n_samples": int(len(is_clean_lst)),
"n_poisoned_flagged": len(poisoned),
"poisoned_indices_sample": poisoned[:50]}
print(json.dumps(report, indent=2))
return report
def main():
ap = argparse.ArgumentParser(description="Data/model poisoning detection agent")
sub = ap.add_subparsers(dest="mode", required=True)
pi = sub.add_parser("integrity", help="Verify weight digests / flag unsafe formats")
pi.add_argument("--model-dir", required=True)
pi.add_argument("--expected-sha256", help="Expected safetensors SHA-256")
pl = sub.add_parser("labels", help="Cleanlab label-issue detection")
pl.add_argument("--labels", required=True, help=".npy integer labels")
pl.add_argument("--pred-probs", required=True, help=".npy out-of-sample probs")
pa = sub.add_parser("activations", help="ART activation-clustering detection")
pa.add_argument("--model", required=True, help="Keras .h5 model")
pa.add_argument("--x", required=True, help=".npy training inputs")
pa.add_argument("--y", required=True, help=".npy training labels")
args = ap.parse_args()
if args.mode == "integrity":
run_integrity(args)
elif args.mode == "labels":
run_labels(args)
elif args.mode == "activations":
run_activations(args)
if __name__ == "__main__":
main()