Subdomain Deep Dive

This guide explains the technical foundations behind each subdomain enumeration technique used in reconFTW.

Execution Pipeline

reconFTW executes subdomain enumeration in a specific order based on data dependencies:

Phase 1: Passive Sources (no target contact)
├── sub_passive   → API queries (subfinder, github-subdomains)
└── sub_crt       → Certificate Transparency logs

Phase 2: DNS Resolution (validates passive results)
├── sub_active    → Resolve collected subdomains with puredns
├── sub_noerror   → DNSSEC NOERROR response analysis
└── sub_dns       → DNS record extraction

Phase 3: Post-Resolution Analysis (requires resolved subdomains)
├── sub_tls       → TLS certificate extraction from live hosts
└── sub_analytics → Google Analytics ID correlation

Phase 4: Bruteforce (resource intensive)
├── sub_brute         → DNS bruteforce with wordlists
├── sub_permut        → Permutation generation (gotator/ripgen)
├── sub_regex_permut  → Regex-based pattern permutations
└── sub_ia_permut     → AI-powered permutation generation

Phase 5: Recursive (multiplies work)
├── sub_recursive_passive → Passive enum on discovered subdomains
├── sub_recursive_brute   → Bruteforce on discovered subdomains
└── sub_scraping          → Web scraping for subdomain extraction

Passive Techniques

sub_passive

Queries multiple APIs and databases that aggregate DNS data globally.

Tools used:

subfinder - Queries 50+ passive sources
github-subdomains - Searches GitHub code for subdomains
gitlab-subdomains - Searches GitLab code for subdomains

Data sources include:

SecurityTrails, VirusTotal, AlienVault OTX
Shodan, Censys, BinaryEdge
Wayback Machine, Common Crawl
DNS aggregators (DNSDB, PassiveTotal)

Why multiple sources: Each source has different coverage. A subdomain may appear in VirusTotal but not SecurityTrails. Combining sources maximizes discovery.

sub_crt

Queries Certificate Transparency (CT) logs via crt.sh API.

How CT works:

Certificate Authorities must log every issued certificate to public logs
Logs are append-only and publicly queryable
Certificates contain Subject Alternative Names (SANs) listing all domains

What it reveals:

Internal subdomains (admins often request certs for internal tools)
Wildcard certificate patterns
Historical infrastructure (old certificates remain in logs)

Time fencing (DNS_TIME_FENCE_DAYS):

# Filter to last 90 days
DNS_TIME_FENCE_DAYS=90

CT logs contain years of historical data. Time fencing filters results to recent certificates, reducing dead subdomain noise.

Implementation:

crt -s -json -l "${CTR_LIMIT}" "$domain" \
    | jq -r --arg cutoff "$cutoff_date" \
      '[.[] | select(.not_before >= $cutoff)] | .[].subdomain'

Active DNS Resolution

sub_active

Resolves all collected subdomains to filter non-existent ones.

Tool used: puredns with massdns backend

Process:

Combine all passive results into single file
Resolve against trusted resolvers
Filter wildcards using random probe technique
Output only subdomains that resolve

Wildcard handling:

puredns resolve input.txt \
    --wildcard-tests 30 \
    --wildcard-batch 1500000 \
    -r "$resolvers_trusted"

Why wildcard detection matters: A wildcard DNS record (*.example.com) returns a valid response for ANY subdomain. Without filtering, you get thousands of false positives.

sub_noerror

Exploits DNSSEC NOERROR responses to enumerate subdomains.

Technical background:

Normal DNS returns NXDOMAIN for non-existent domains
Some DNSSEC-signed zones return NOERROR instead (black lies technique)
This allows enumeration via response code analysis

When useful:

Targets with DNSSEC-signed zones
Can find subdomains not in any passive source

sub_dns

Extracts additional information from DNS records.

Records analyzed:

A/AAAA → IP addresses
CNAME → Alias targets (potential takeover)
MX → Mail servers
TXT → SPF, DKIM, verification records
NS → Nameservers

Post-Resolution Techniques

sub_tls

Connects to discovered hosts and extracts TLS certificate SANs.

Tool used: tlsx

Ports checked:

TLS_PORTS="21,22,25,80,110,135,143,261,443,465,563,587,636,853,990,993,995,..."

What it finds:

Additional domains sharing the same certificate
Internal names in SAN fields
Related infrastructure

Why run after resolution: Requires IP addresses from resolved subdomains. Must run after sub_active.

sub_analytics

Finds domains sharing the same Google Analytics or Tag Manager ID.

Tool used: AnalyticsRelationships

How it works:

Extract analytics IDs from resolved web pages
Query databases for other domains with same ID
Related domains likely owned by same organization

Why run after resolution: Requires web-accessible hosts to extract analytics IDs.

Bruteforce Techniques

sub_brute

DNS bruteforce using wordlists.

Tool used: puredns with optimized wordlists

Wordlist strategy:

Default: ~100K common subdomain names
DEEP mode: ~1M+ extended wordlist

When useful:

New subdomains not yet indexed by passive sources
Internal naming conventions (dev, staging, prod)
Non-web services without certificates

sub_permut

Generates permutations based on discovered subdomains.

Tool used: gotator or ripgen

Example:

Found: api-v1.example.com
Generates:
  - api-v2.example.com
  - api-v3.example.com  
  - api-dev.example.com
  - api-staging.example.com
  - api-prod.example.com

Configuration:

PERMUTATIONS_OPTION=gotator  # Deeper but slower
PERMUTATIONS_OPTION=ripgen   # Faster but less thorough

sub_regex_permut

Uses regex pattern learning to generate permutations.

Tool used: regulator

How it works:

Analyze discovered subdomain patterns
Learn regex rules from patterns
Generate new candidates matching rules

sub_ia_permut

AI-powered permutation generation.

Tool used: subwiz

Approach: Uses machine learning models trained on subdomain patterns to generate likely candidates.

Deep Wildcard Detection

Standard wildcard detection only checks the root level (*.example.com). Enterprise environments often have nested wildcards.

Problem example:

*.na45.salesforce.com  → All subdomains under na45 resolve
*.api.prod.example.com → All API subdomains in prod resolve

How deep_wildcard_filter() works:

Iteration 1:
  Input: a.b.c.example.com, x.b.c.example.com, y.c.example.com
  Test:  random123.b.c.example.com
  Result: Resolves → b.c.example.com is wildcard
  Filter: Remove all *.b.c.example.com

Iteration 2:
  Input: y.c.example.com
  Test:  random456.c.example.com
  Result: Does not resolve → c.example.com is not wildcard
  Filter: Keep y.c.example.com

Final: y.c.example.com

Implementation details:

Maximum 5 iterations
Uses dnsx with trusted resolvers
Random probe: 12-character alphanumeric string
Results saved to subdomains/wildcards_detected.txt

Sensitive Domain Exclusion

The _is_sensitive_domain() function checks domains against patterns in config/sensitive_domains.txt.

Pattern format:

*.gov       → Matches any .gov domain
*.gov.*     → Matches any .gov.XX domain
*.mil       → Matches military domains
*.edu       → Matches educational domains

Matching logic:

# Wildcard pattern (*.gov)
if [[ "$domain" == *".$suffix" ]] || [[ "$domain" == "$suffix" ]]; then
    return 0  # Is sensitive
fi

Use case: When scanning wildcard scopes, prevents enumeration of government or military subdomains that may be part of the target's acquired assets.

Output Files

File

Description

subdomains/subdomains.txt

Final deduplicated subdomain list

subdomains/subdomains_new.txt

New subdomains since last run (incremental)

subdomains/wildcards_detected.txt

Detected wildcard parent domains

.tmp/passive_subs.txt

Raw passive enumeration results

.tmp/crtsh_subs.txt

Certificate transparency results

.tmp/subs_no_resolved.txt

Subdomains before resolution

Performance Considerations

Technique

Speed

Resource Usage

When to Disable

sub_passive

Fast

Low (API calls)

Never

sub_crt

Fast

Low

Never

sub_active

Medium

Never

sub_brute

Slow

High

Large targets

sub_permut

Slow

High

Large targets

sub_recursive_*

Very slow

Very high

Most scans

DEEP_WILDCARD_FILTER

Medium

Small targets

Parallel mode (--parallel): Runs independent functions concurrently. Phases maintain dependencies:

Passive functions run in parallel
Wait for passive to complete
Active functions run in parallel
Post-active functions run after resolution
Bruteforce runs with limited parallelism (resource intensive)

PreviousSubdomain Module NextWeb Analysis Module

Last updated 1 month ago

Good afternoon

hashtagExecution Pipeline

hashtagPassive Techniques

hashtagsub_passive

hashtagsub_crt

hashtagActive DNS Resolution

hashtagsub_active

hashtagsub_noerror

hashtagsub_dns

hashtagPost-Resolution Techniques

hashtagsub_tls

hashtagsub_analytics

hashtagBruteforce Techniques

hashtagsub_brute

hashtagsub_permut

hashtagsub_regex_permut

hashtagsub_ia_permut

hashtagDeep Wildcard Detection

hashtagSensitive Domain Exclusion

hashtagOutput Files

hashtagPerformance Considerations

Execution Pipeline

Passive Techniques

sub_passive

sub_crt

Active DNS Resolution

sub_active

sub_noerror

sub_dns

Post-Resolution Techniques

sub_tls

sub_analytics

Bruteforce Techniques

sub_brute

sub_permut

sub_regex_permut

sub_ia_permut

Deep Wildcard Detection

Sensitive Domain Exclusion

Output Files

Performance Considerations