WhatWeb Integration with DefectDojo

WhatWeb Integration with DefectDojo

WhatWeb is an open source website fingerprinting tool maintained by Andrew Horton and Brendan Coles and licensed under GPLv2. It requests a URL and uses its plugin library (over 1,800 plugins, per the project) to identify what the site runs: web server and version, frameworks, content management systems, JavaScript libraries, analytics, page title, and meta generator tags. Aggression levels control how many extra requests it sends, and --log-json writes the JSON log that DefectDojo imports.

WhatWeb Integration with DefectDojo

We use WhatWeb as a cheap first pass over every web property we own. It tells us which hosts expose a server version banner, which sites still run an old CMS release, and which ones nobody remembers deploying. Importing WhatWeb JSON into DefectDojo turns each scanned URL into one tidy inventory finding on the right Asset, so the disclosed versions sit beside the vulnerability findings for that site and change history is kept between scans.

Why WhatWeb Matters

You cannot patch software you do not know is running, and technology drift on websites is common.

• It identifies servers, frameworks, and CMS versions from the outside, the same way an attacker would.
• It runs passively at aggression level 1, or sends extra confirming requests at level 3 and above.
• It covers any URL it can reach, which makes it useful for discovering forgotten or shadow sites.
• Disclosed version strings are often the first clue that a site needs patching.

Advantages of This Integration

What DefectDojo adds to raw WhatWeb output:

• One finding per URL, not per plugin. WhatWeb fingerprints anything it reaches, and even a 404 page yields several plugin matches. DefectDojo groups every detected technology for a URL into one finding instead of flooding the Test.
• Network details kept separate. The IP and Country plugins describe the network WhatWeb used, not the site. They are listed under their own Network heading so nobody reads Country: RESERVED as something the site runs.
• Honest severity. Every finding is Info. Knowing a site runs a given web server version is inventory, not a weakness, and DefectDojo treats it the same way it treats discovered paths and open ports.
• Endpoint per target. The scanned URL becomes the finding's endpoint, so inventory can be filtered by host alongside DAST results for the same site.
• Stable deduplication. Findings are matched on title and endpoint, so a rescan of an unchanged site does not create a new finding just because a detail in the description changed.

How This Integration Works

DefectDojo imports WhatWeb output with the WhatWeb Scan scan type, using UI Import, API Import, or Universal Importer in DefectDojo Pro.

1. Run WhatWeb with JSON logging. Only the --log-json output is parsed:

whatweb --log-json=whatweb.json --no-errors https://target.example.com/

Add -a 3 or higher if you want WhatWeb to send extra requests to confirm its guesses. WhatWeb appends to an existing JSON log rather than overwriting it, so delete the file or use a new name between runs. Otherwise one report holds two scans and imports as duplicates.

To cover many sites in one run, pass several URLs on the command line or a file of targets with --input-file. Each target in the log becomes its own finding, so a single import can describe an entire group of sites. If those sites belong to different teams, split the targets into separate runs and import each log into the Asset that team owns, which keeps assignment and reporting clean.

2. Import the log. In the UI, open the Engagement, choose Import Scan Results, select WhatWeb Scan, and upload the file. Through the API, in Community Edition or DefectDojo Pro:

curl "https://YOUR_INSTANCE/api/v2/import-scan/" 
  -H "Authorization: Token $DD_API_TOKEN" 
  -F "scan_type=WhatWeb Scan" 
  -F "file=@whatweb.json" 
  -F "product_name=marketing-sites" 
  -F "engagement_name=Web Inventory" 
  -F "auto_create_context=true"

With Universal Importer in DefectDojo Pro:

universal-importer import 
  --defectdojo-url "https://YOUR_INSTANCE.cloud.defectdojo.com/" 
  --scan-type "WhatWeb Scan" 
  --report-path "./whatweb.json" 
  --product-name "marketing-sites" 
  --engagement-name "Web Inventory" 
  --auto-create-context

3. Reimport on a schedule. Reimport later logs into the same Test. URLs that stop answering drop out as mitigated, and new ones appear as new findings.

An unreachable target produces an empty log. Any URL that answers, including with a 404, is fingerprinted and produces a finding.

Data Granularity: What Gets Imported

The description is left out of the hash on purpose. It records what the scan saw at that moment, which can change between scans of an unchanged site.

Use Cases

Attack surface inventory: A security team runs WhatWeb across every domain in its DNS zones each week. DefectDojo holds one inventory finding per URL, and new findings flag sites that appeared since the last run.

Version disclosure cleanup: Engineers search finding descriptions for server banners and CMS versions, then remove version headers or upgrade. The next import shows the updated technology list.

Pairing with vulnerability scans: WhatWeb findings sit next to DAST and infrastructure findings in the same Asset. When a new advisory drops for a framework, the team checks which sites WhatWeb saw running it.

Pre-assessment reconnaissance: Before a penetration test, testers import WhatWeb results into the engagement so the whole team sees the same technology map.

Operational Tips

• Use a fresh output file for each run, because --log-json appends.
• Keep WhatWeb findings in their own Engagement or tag them (for example tags=inventory) so Info-level inventory does not clutter vulnerability metrics.
• Leave severity at Info and do not set SLAs on these findings. Raise a separate finding, or a note, when a disclosed version maps to a known CVE.
• Choose aggression deliberately. Level 1 is passive; higher levels send more requests and are better saved for systems you are authorized to test actively.
• Scan only targets you own or are authorized to assess.
• Use reimport rather than fresh imports so sites that disappear are mitigated instead of lingering.
• Check the Status line in each description. A URL that answers 404 or redirects is still fingerprinted, which can mean a stale DNS record or a parked host worth cleaning up.