oscp/Necronomicon/OSINT/Apex Domains and Subdomains.md

• Scraping

  • Tons of databases and data projects exist to catalogue related URLs and domains
  • Google Dorking
    • Google a domain, then progressively subtract known subdomains
      1. site:twitch.tv -www.twitch.tv
      2. site:twitch.tv -www.twitch.tv -watch.twitch.tv
      3. site:twitch.tv -www.twitch.tv -watch.twitch.tv -dev.twitch.tv
      4. ...
  • Amass
    • Will automatically extract subdomain data from tons of sources and optionally brute force subdomains as well.
    • Also groups scraped domains to ASNs, owners, and IP ranges
    • Make sure to review the API Keys page to make your Amass as much as it can be
      • amass enum –list
      • amass enum -list | grep -v "\*"
      • Put the keys in ~/.config/amass/config.ini
  • Subfinder
    • Similar to Amass - use both and concat | uniq the output
  • BBOT
    • Huge tool with subdomain scraping, brute force, web spidering, and more.
    • The output is a file at /root/.bbot/scans/{scan_name}/
      • cat /root/.bbot/scans/{scan_name}/output.txt | grep -F '[DNS_NAME]’ | awk '{print $2}'

• Subdomain brute force

  • Very slow - try using threaded tools that will use multiple DNS resolvers
  • PureDNS
    • Wrapper around MassDNS and adds features
    • puredns bruteforce <subdomain_list> tesla.com -r resolvers.txt
  • DNS resolvers: https://github.com/trickest/resolvers
  • Permutation/Alteration scanning - predict other subdomain names based on patterns
    • altdns - generate permutations, but doesn't attempt to resolve. Use with PureDNS
    • dnsgen - generate permutations, but doesn't attempt to resolve. Use with PureDNS
      • cat <file_of_subdomains.txt> | dnsgen - | puredns resolve --resolvers resolvers.txt

• Shodan

  • Shosubgo
    • go run main.go -d <target_domain>
  • Shodan from CLI is better for large targets - NahamSec has a great video on this and parsing output
    • NahamSec - Asset Discovery Using Shodan

• Acquisitions

  • Crunchbase
    • Business information about acquisitions and mergers - new apex domains that belong to your target for enumeration, phishing, etc.
    • Companies often don't force newly acquired companies to change to their infrastructure immediately.
    • Also gives:
      • information on company leadership and employees - even past employees
      • Tech stack info
      • Annual revenue
      • Events
      • Recent news
      • Other fantastic contextual data
  • OCCRP
    • global archive of research material for investigative reporting. They keep track of 414 million public entities and parse over 252 discrete datasets in more than 141 countries.
    • When searching for a company, find the entry that comes up, closely related to your company and associated with the dataset "US SEC CorpWatch.“
      • Look for one with a substantial list of assets
    • Provides a list of acquisitions and investments by your target

• Reverse WHOIS

  • The purpose of this is to search WHOIS data to hopefully find other apex domains owned by the target.
  • For example, searching WHOIS data for the "Organization Name" or "Registrant Email" of the target
  • https://whoxy.com - cheapest place for access to reverse WHOIS and WHOIS data in general.
    • API - 1000 queries for $10
      • https://api.whoxy.com/?key=xxxxx&reverse=whois&keyword=google&mode=domains

• Using AI for recon

  • Current dataset cuts off in 2021; all results should be rigorously verified
  • Acquisitions
    • "What can you tell me about Tesla's acquisitions?"
  • Subdomains
    • SubreconGPT
      • Script that accepts a list of subdomains and for each one, it will have GPT4 guess similar/related subdomains and check whether they resolve

• Linked Discovery

  • This essentially involves spidering pages at the known domains of the target to discover more related domains.
  • BurpSuite:
    1. Turn off passive scanning
    2. Set forms auto to submit (if you’re feeling frisky)
    3. Set scope to advanced control and use “keyword” of target name (not a normal FQDN)
    4. Walk+browse main site, then spider all hosts recursively!
    5. Profit
    • To export the found domains/subdomains:
      1. Select all hosts in the site tree
      2. In PRO ONLY right click the selected hosts
      3. Go to “Engagement Tools” -> “Analyze target”
      4. Save report as an html file
      5. Copy the hosts from the “Target” section
  • GoSpider
  • hakrawler

• Ad & Analytics

  • You can also glean related domains and subdomains by looking at a target’s ad/analytics tracker codes. Many sites use the same codes across all their domains. Google analytics and New Relic codes are the most common.
  • BuiltWith
    • Example: https://builtwith.com/relationships/twitch.tv
    • [getrelationship.py](https://raw.githubusercontent.com/ m4ll0k/Bug-Bounty- Toolz/master/getrelationship.py)
      • Will search BuiltWith from the command line with the help of your session token.

• Discover cloud endpoints behind Cloudflare, Akamai, etc.

  • Pull CNAME records from domains
    • getent hosts <domain>
  • Analyze public IPs and cross-reference with IP ranges of various services like AWS
    • DNSCharts