Fickling Integration with DefectDojo

Fickling Integration with DefectDojo

Fickling is an open source tool from Trail of Bits for decompiling, statically analyzing, and rewriting Python pickle files. Pickle is Python's native serialization format, and it is still common for distributing machine learning models, which matters because loading a pickle can execute code embedded in it. Fickling inspects the pickle bytecode without loading it, returns a safety verdict for the file along with the observations behind that verdict, and can write the result as JSON for DefectDojo to import.

Fickling Integration with DefectDojo

We added Fickling to the pipeline that pulls third-party models into our environment, and DefectDojo is where its verdicts get acted on. A Fickling JSON report on its own says whether one file looked dangerous. Imported into DefectDojo, each observation becomes a Finding on the Asset that owns the model, with a severity drawn from Fickling's verdict, a mitigation telling engineers not to unpickle the file, and a record of who reviewed it and when. When a model source is replaced or re-serialized, the next import shows whether the problem actually went away.

Why Fickling Matters

Model files are code as much as they are data. Teams download them from public hubs, internal registries, and vendors, often with less scrutiny than a new library would get.

• Unpickling an untrusted file can run arbitrary code with the permissions of whatever process loads it, which is often a training job or inference service with broad access.
• Fickling reads the pickle opcodes statically, so the analysis itself does not execute the payload.
• It reports specific observations, such as imports of functions that are unsafe to call during unpickling, or variables assigned from calls and never used, instead of a single yes or no.
• Its verdict scale distinguishes overtly malicious files from merely suspicious ones, which gives a reviewer somewhere to start.

Advantages of This Integration

What we get from routing Fickling reports through DefectDojo:

• Per-observation Findings. Each item in Fickling's detailed results becomes its own Finding carrying the file's verdict, so a malicious pickle shows every reason it was flagged rather than collapsing into one result.
• Severity that maps to SLAs. Fickling's verdicts translate into Critical, High, and Medium, so model files fall under the same remediation timelines as every other source of risk.
• Deduplication across rescans. The Fickling Scan type deduplicates on vuln_id_from_tool (the Fickling check name) and severity, so scanning the same model again doesn't duplicate its Findings.
• Ownership and audit trail. Findings can be assigned to the team that requested the model, annotated with the source it came from, and risk-accepted with an expiration if a model must be used while a safer format is prepared.
• One view of supply chain risk. Model file findings sit next to SCA, SAST, and container results for the same Asset.

How This Integration Works

DefectDojo imports Fickling results with the Fickling Scan scan type.

1. Run Fickling with JSON output. The JSON file is only written when the safety check is also requested:

fickling --check-safety --json-output fickling.json model.pkl

2. Import it. In the UI, open the Engagement, choose Import Scan Results, select Fickling Scan, and upload the file. For automation, use the API in Community Edition or DefectDojo Pro:

curl "https://YOUR_INSTANCE/api/v2/import-scan/" 
  -H "Authorization: Token $DD_API_TOKEN" 
  -F "scan_type=Fickling Scan" 
  -F "file=@fickling.json" 
  -F "product_name=recommendation-model" 
  -F "engagement_name=Model Intake" 
  -F "test_title=model.pkl" 
  -F "auto_create_context=true"

DefectDojo Pro users can do the same with Universal Importer:

universal-importer import 
  --defectdojo-url "https://YOUR_INSTANCE.cloud.defectdojo.com/" 
  --scan-type "Fickling Scan" 
  --report-path "./fickling.json" 
  --product-name "recommendation-model" 
  --engagement-name "Model Intake" 
  --auto-create-context

3. Keep one report per file. Fickling's JSON does not record which file it analyzed, so the resulting Findings have no file path. Importing one report per pickle, with a Test title or tag naming the file, is how you keep them apart.

Data Granularity: What Gets Imported

Severity mapping: OVERTLY_MALICIOUS and LIKELY_OVERTLY_MALICIOUS become Critical, LIKELY_UNSAFE becomes High, and SUSPICIOUS becomes Medium. An unrecognized verdict is also imported as Medium. A LIKELY_SAFE verdict produces no Findings at all. If Fickling returns a verdict with no itemized results, the parser creates a single Finding titled Pickle flagged as <verdict> so the verdict isn't lost.

Use Cases

Gating model intake: An ML platform team requires every externally sourced model to pass through a Fickling check before it reaches the shared registry. Each report is imported into DefectDojo under the requesting team's Asset, and a Critical Finding blocks promotion until someone reviews it.

Auditing an existing model store: A security engineer scans every pickle-based artifact in an internal bucket, one report per file, and imports them into a single Engagement. The result is a sortable inventory of which models carry unsafe imports, with owners assigned from the Asset structure.

Tracking migration to safer formats: A team re-serializing models into a format that cannot carry executable code rescans each one afterward. Reimporting into the same Test closes the old Findings and leaves a record that the conversion happened.

Operational Tips

• Always pass --check-safety together with --json-output. Without the safety check, Fickling does not write the JSON file and there is nothing to import.
• Name the Test after the file (test_title) or tag it with the model name and version. With no file path on the Findings, that context is the only way to tell two models apart later.
• Treat a LIKELY_SAFE result as an absence of evidence. The parser imports nothing for it, which keeps the queue clean, but it does not prove the file is harmless.
• Deduplication is keyed only on the check name and severity, so two different pickles with the same problem can be flagged as duplicates of each other within an Asset. If you scan many models under one Asset, give each model its own Engagement and import with deduplication_on_engagement=true.
• Configure SLAs for Critical findings with this source in mind. A Critical from Fickling usually means "do not load this file," which is a faster response than a typical patch cycle.
• When a model genuinely needs a suspicious construct, document the reasoning in a note and risk-accept with an expiration rather than marking it a false positive.