The 7 Best Subdomain finder tools

3
4007

DNS enumeration is considered one of the most important information-gathering techniques. Penetration testers spent a lot of time to find all available subdomains of a target, as they unlock multiple new attack opportunities.

There are plenty of tools out there that can make your life easier. However, as time passes, the tool number is increased so much that it triggers a headache in a junior pen-tester.

The purpose of this article is to present the most common sub-domain discovery methods and then evaluate as many tools as possible in real-life scenarios.

How subdomain finder works

Each tool uses different methods to enumerate subdomains. The article will not cover in-depth each method. Methods that depend on external input will be used in a fairly way.  For example, all tools will be tested with the same wordlist for brute-forcing.

The most common methods for subdomain enumeration are :

Search Engines

One of the most common subdomain enumeration techniques is via using search engines (Google, Bing, etc.). All search engines use thousands of spider bots to crawl the internet constantly and keep track of billion of subdomains.

Google is powerful when enumerating subdomains like this :   site:example_domain.com

google subdomain finder
Google as a Subdomain Finder Tool

Many times the results are filled with the same subdomain.

In case you want to exclude similar subdomain results  use the minus flag  :                     -site:subdomain.example.com .

Google Exclude Results
How to exclude Subdomains from Google Results

Search engine enumeration is passive and not a single packet is sent to the target!

DNS Historical Data

Historical data can be a valuable source of old DNS records. Securitytrails.com holds over a 3.4 trillion DNS records and it also has a separate subdomain enumeration function.

Security Trails
SecurityTrails DNS historical data enumeration

DNS Records

DNS servers create a DNS record to provide important information about a domain or hostname, particularly its current IP address. The most common DNS record types are:

  • Address Mapping record (A Record)—also known as a DNS host record, stores a hostname and its corresponding IPv4 address.
  • IP Version 6 Address record (AAAA Record)—stores a hostname and its corresponding IPv6 address.
  • Canonical Name record (CNAME Record)—can be used to alias a hostname to another hostname. When a DNS client requests a record that contains a CNAME, which points to another hostname, the DNS resolution process is repeated with the new hostname.
  • Mail exchanger record (MX Record)—specifies an SMTP email server for the domain, used to route outgoing emails to an email server.
  • Name Server records (NS Record)—specifies that a DNS Zone, such as “example.com” is delegated to a specific Authoritative Name Server, and provides the address of the name server.
  • Reverse-lookup Pointer records (PTR Record)—allows a DNS resolver to provide an IP address and receive a hostname (reverse DNS lookup).
  • Certificate record (CERT Record)—stores encryption certificates—PKIX, SPKI, PGP, and so on.
  • Service Location (SRV Record)—a service location record, like MX but for other communication protocols.
  • Text Record (TXT Record)—typically carries machine-readable data such as opportunistic encryption, sender policy framework, DKIM, DMARC, etc.
  • Start of Authority (SOA Record)—this record appears at the beginning of a DNS zone file, and indicates the Authoritative Name Server for the current DNS zone, contact details for the domain administrator, domain serial number, and information on how frequently DNS information for this zone should be refreshed.

All these records can easily be retrieved with an online service or via the Linux command-line tool dig as shown below :

Dig can easily find all youtube’s nameserver by using the NS parameter, short  flag adjusts the answer size :

# dig youtube.com NS +short
ns1.google.com.
ns2.google.com.
ns3.google.com.
ns4.google.com.

Now we can query youtube’s nameserver for all DNS records with ANY parameter

# dig @ns3.google.com youtube.com ANY

; <<>> DiG 9.11.5-P1-1-Debian <<>> @ns3.google.com youtube.com ANY
; (2 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 61955
;; flags: qr aa rd; QUERY: 1, ANSWER: 16, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;youtube.com.                   IN      ANY

;; ANSWER SECTION:
youtube.com.            300     IN      AAAA    2a00:1450:4017:800::200e
youtube.com.            300     IN      A       172.217.17.174
youtube.com.            600     IN      MX      10 aspmx.l.google.com.
youtube.com.            600     IN      MX      30 alt2.aspmx.l.google.com.
youtube.com.            3600    IN      TXT     "facebook-domain-verification=64jdes7le4h7e7lfpi22rijygx58j1"
youtube.com.            345600  IN      NS      ns3.google.com.
youtube.com.            600     IN      MX      50 alt4.aspmx.l.google.com.
youtube.com.            600     IN      MX      40 alt3.aspmx.l.google.com.
youtube.com.            3600    IN      TXT     "google-site-verification=OQz60vR-YapmaVrafWCALpPyA8eKJKssRhfIrzM-DJI"
youtube.com.            345600  IN      NS      ns1.google.com.
youtube.com.            345600  IN      NS      ns4.google.com.
youtube.com.            600     IN      MX      20 alt1.aspmx.l.google.com.
youtube.com.            60      IN      SOA     ns1.google.com. dns-admin.google.com. 271534513 900 900 1800 60
youtube.com.            3600    IN      TXT     "v=spf1 include:google.com mx -all"
youtube.com.            86400   IN      CAA     0 issue "pki.goog"
youtube.com.            345600  IN      NS      ns2.google.com.

;; Query time: 78 msec
;; SERVER: 216.239.36.10#53(216.239.36.10)
;; WHEN: Sat Sep 28 13:19:16 DST 2019
;; MSG SIZE  rcvd: 543

 

DNS Zone Transfers

DNS service is critical for the whole internet and plenty of DNS records are replicated in multiple servers for redundancy purposes. The easiest way to copy records between DNS servers is via zone transfers.

Zone transfers should only be allowed between trusted hosts. Do not forget to set the appropriate IPs in the access lists at # /etc/named.conf  of your DNS server.

If a DNS server is misconfigured then the attacker may get confidential information of your internal or external hosts. The following one-liner can check if the first DNS server of the target domain is vulnerable.

#target=zonetransfer.me;ns=$(dig @1.1.1.1 $target NS +short|head -1);dig @$ns $target AXFR

; <<>> DiG 9.11.5-P1-1-Debian <<>> @nsztm2.digi.ninja. zonetransfer.me AXFR
; (1 server found)
;; global options: +cmd
zonetransfer.me.        7200    IN      SOA     nsztm1.digi.ninja. robin.digi.ninja. 2019031901 172800 900 1209600 3600
zonetransfer.me.        300     IN      HINFO   "Casio fx-700G" "Windows XP"
zonetransfer.me.        301     IN      TXT     "google-site-verification=tyP28J7JAUHA9fw2sHXMgcCC0I6XBmmoVi04VlMewxA"
zonetransfer.me.        7200    IN      MX      0 ASPMX.L.GOOGLE.COM.
zonetransfer.me.        7200    IN      MX      10 ALT1.ASPMX.L.GOOGLE.COM.
zonetransfer.me.        7200    IN      MX      10 ALT2.ASPMX.L.GOOGLE.COM.
zonetransfer.me.        7200    IN      MX      20 ASPMX2.GOOGLEMAIL.COM.
zonetransfer.me.        7200    IN      MX      20 ASPMX3.GOOGLEMAIL.COM.
zonetransfer.me.        7200    IN      MX      20 ASPMX4.GOOGLEMAIL.COM.
zonetransfer.me.        7200    IN      MX      20 ASPMX5.GOOGLEMAIL.COM.
zonetransfer.me.        7200    IN      A       5.196.105.14
zonetransfer.me.        7200    IN      NS      nsztm1.digi.ninja.
zonetransfer.me.        7200    IN      NS      nsztm2.digi.ninja.
_sip._tcp.zonetransfer.me. 14000 IN     SRV     0 0 5060 www.zonetransfer.me.
14.105.196.5.IN-ADDR.ARPA.zonetransfer.me. 7200 IN PTR www.zonetransfer.me.
asfdbauthdns.zonetransfer.me. 7900 IN   AFSDB   1 asfdbbox.zonetransfer.me.
asfdbbox.zonetransfer.me. 7200  IN      A       127.0.0.1
asfdbvolume.zonetransfer.me. 7800 IN    AFSDB   1 asfdbbox.zonetransfer.me.
canberra-office.zonetransfer.me. 7200 IN A      202.14.81.230
cmdexec.zonetransfer.me. 300    IN      TXT     "; ls"
contact.zonetransfer.me. 2592000 IN     TXT     "Remember to call or email Pippa on +44 123 4567890 or pippa@zonetransfer.me when making DNS changes"
dc-office.zonetransfer.me. 7200 IN      A       143.228.181.132
deadbeef.zonetransfer.me. 7201  IN      AAAA    dead:beaf::
dr.zonetransfer.me.     300     IN      LOC     53 20 56.558 N 1 38 33.526 W 0.00m 1m 10000m 10m
DZC.zonetransfer.me.    7200    IN      TXT     "AbCdEfG"
email.zonetransfer.me.  2222    IN      NAPTR   1 1 "P" "E2U+email" "" email.zonetransfer.me.zonetransfer.me.
email.zonetransfer.me.  7200    IN      A       74.125.206.26
home.zonetransfer.me.   7200    IN      A       127.0.0.1
Info.zonetransfer.me.   7200    IN      TXT     "ZoneTransfer.me service provided by Robin Wood - robin@digi.ninja. See http://digi.ninja/projects/zonetransferme.php for more information."
internal.zonetransfer.me. 300   IN      NS      intns1.zonetransfer.me.
internal.zonetransfer.me. 300   IN      NS      intns2.zonetransfer.me.
intns1.zonetransfer.me. 300     IN      A       81.4.108.41
intns2.zonetransfer.me. 300     IN      A       52.91.28.78
office.zonetransfer.me. 7200    IN      A       4.23.39.254
ipv6actnow.org.zonetransfer.me. 7200 IN AAAA    2001:67c:2e8:11::c100:1332
owa.zonetransfer.me.    7200    IN      A       207.46.197.32
robinwood.zonetransfer.me. 302  IN      TXT     "Robin Wood"
rp.zonetransfer.me.     321     IN      RP      robin.zonetransfer.me. robinwood.zonetransfer.me.
sip.zonetransfer.me.    3333    IN      NAPTR   2 3 "P" "E2U+sip" "!^.*$!sip:customer-service@zonetransfer.me!" .
sqli.zonetransfer.me.   300     IN      TXT     "' or 1=1 --"
sshock.zonetransfer.me. 7200    IN      TXT     "() { :]}; echo ShellShocked"
staging.zonetransfer.me. 7200   IN      CNAME   www.sydneyoperahouse.com.
alltcpportsopen.firewall.test.zonetransfer.me. 301 IN A 127.0.0.1
testing.zonetransfer.me. 301    IN      CNAME   www.zonetransfer.me.
vpn.zonetransfer.me.    4000    IN      A       174.36.59.154
www.zonetransfer.me.    7200    IN      A       5.196.105.14
xss.zonetransfer.me.    300     IN      TXT     
zonetransfer.me.        7200    IN      SOA     nsztm1.digi.ninja. robin.digi.ninja. 2019031901 172800 900 1209600 3600
;; Query time: 190 msec
;; SERVER: 34.225.33.2#53(34.225.33.2)
;; WHEN: Sat Sep 28 14:09:04 DST 2019
;; XFR size: 48 records (messages 1, bytes 1903)

Do not forget to test all your DNS servers! 

The expected output of a correct-secure configuration is the following :

target=youtube.com;ns=$(dig @1.1.1.1 $target NS +short|head -1);dig @$ns $target AXFR +short
; Transfer failed.

The best way to tackle zone transfers is via enabling transactions signatures (TSIG).

Brute Force Subdomains

Many tools use brute force to enumerate subdomains. Grab your wordlist, and sequentially try to resolve all combinations. This method can be used recursively and on top of all other methods to detect subdomains of already found subdomains.

Some administrators may think that if they create a wildcard domain ( an A Record of *.domain.com) that they will be protected, as all requests will be successfully answered during the attack. This is misjudgment as the attacker will filter the IP of a 100% never exist domain from the results.

Brute Force tools are as powerful as the used wordlist. All must-have DNS subdomain discovery wordlists can be found on Daniel Miessler’s Seclists.

subdomains seclist
Subdomains SecLists

 

Reverse DNS Lookup IPs

Reverse DNS lookup is the reverse of a forward DNS lookup. A reverse DNS lookup returns the hostname when you provide an IP.

Forwards DNS:  dns.google.com resolved to 8.8.8.8

Reverse  DNS:  8.8.8.8 resolved to dns.google.com

Pointer (PTR) records provide what is known as “reverse DNS”. PTR records assign IP addresses to a hostname instead of mapping a hostname to an IP address.

Many times the PTR records will not be as useful as you think. Once again  dig  with   -x flag  is our tool,  we will do fDNS to youtube.com and then try to rDNS the resolved IP.

# dig youtube.com +short
172.217.20.110
#dig -x 172.217.20.110 +short
ams17s01-in-f14.1e100.net.
fra02s28-in-f14.1e100.net.

It is common for companies to buy ranges of IPs. Attackers may reverse DNS subnets of IPS near all the known public IPs. For example, if YouTube is resolved into 172.217.20.110 then an attacker may try rDNS the 172.217.20.110/24 subnet to enumerate further subdomains.

Many times PTR records are used to increase the trust of a webmail server. System administrators should use the right PTR records according to their needs and keep their number to the minimum value.

Analyze Secure Certificates

The Subject Alternate Name (SAN) of SSL/TLS certificates can be used to extract domains and subdomain names.

The following script can be used to extract such  information via command line :

Installation 
#git clone https://github.com/appsecco/the-art-of-subdomain-enumeration.git
Example SSL certification subdomain enumaration 
# python san_subdomain_enum.py www.cartier.com 
www.userguide.tw.cartier.com
www.userguide.ru.cartier.com
www.userguide.cartier.com
www.cartier.com
www.au.cartier.com
...

Tools

AMASS

The OWASP Amass Project has developed a tool to help information security professionals perform network mapping of attack surfaces and perform external asset discovery using open source information gathering and active reconnaissance techniques.

OWASP Amass Example
OWASP Amass Example

 

Installation

$docker build -t amass https://github.com/OWASP/Amass.git
$docker run -v ~/amass:/amass/ amass enum --list
$docker run -v ~/amass:/amass/ amass enum -brute -w /wordlists/all.txt -d example.com

DNSDUMPSTER

DNSdumpster.com is an online free domain research tool that can discover hosts related to a domain. Finding visible hosts from the attackers’ perspective is an important part of the security assessment process.

 dnsdumpster
DNSDumpster.com

 

FINDSUBDOMAIN

An online tool that allows security engineers to monitor vulnerabilities, identify gaps and weaknesses to prevent attacks. With FindSubdomains.com , pentesters can check the endpoints for vulnerability, including subdomains in the development environment, technical domains open to the public, and much more. The free version shows up to 50 subdomains.

findsubdomain
Findsubdomain.com

PENTEST-TOOLS

Pentest-tools.com is a site that includes multiple penetration testing tools. One of them called “Find Subdomains”  which has two flavors: a) free and b) paid service.

pentest-tools.com
Pentest-Tools.com
Pentest-tools.com Pro features

ASSETFINDER

Assetfinder is a  new tool created by Tom Hudson or Tomnomnom in Go. It finds domains and subdomains potentially related to a given domain by checking several resources online ( facebook,virustotal etc).

Installation

$ go get -u github.com/tomnomnom/assetfinder
$ go install github.com/tomnomnom/assetfinder

Usage

$assetfinder [--subs-only] <domain>
$assetfinder --subs-only example.com
www.example.com
example.com
m.example.com
dev.example.com
products.example.com
support.example.com
privatee191.example.com
fireeye-testevent.example.com
gerendes.cherochk100.example.com
...

SUBLIST3R

Sublist3r is a python tool designed to enumerate subdomains of websites using OSINT. It helps penetration testers and bug hunters collect and gather subdomains for the domain they are targeting. Sublist3r enumerates subdomains using many search engines such as Google, Yahoo, Bing, Baidu and Ask. Sublist3r also enumerates subdomains using Netcraft, Virustotal, ThreatCrowd, DNSdumpster and ReverseDNS.

Usage

root@kali:~# sublist3r -h
usage: sublist3r [-h] -d DOMAIN [-b [BRUTEFORCE]] [-p PORTS] [-v [VERBOSE]]
                 [-t THREADS] [-e ENGINES] [-o OUTPUT]
OPTIONS:
  -h, --help            show this help message and exit
  -d DOMAIN, --domain DOMAIN
                        Domain name to enumerate it s subdomains
  -b [BRUTEFORCE], --bruteforce [BRUTEFORCE]
                        Enable the subbrute bruteforce module
  -p PORTS, --ports PORTS
                        Scan the found subdomains against specified tcp ports
  -v [VERBOSE], --verbose [VERBOSE]
                        Enable Verbosity and display results in realtime
  -t THREADS, --threads THREADS
                        Number of threads to use for subbrute bruteforce
  -e ENGINES, --engines ENGINES
                        Specify a comma-separated list of search engines
  -o OUTPUT, --output OUTPUT
                        Save the results to text file
Example: python /usr/share/sublist3r/sublist3r -d google.com

SUBFINDER

SubFinder is a subdomain discovery tool that discovers valid subdomains for websites by using passive online sources. It has a simple modular architecture and has been aimed as a successor to sublist3r project. SubFinder uses Passive Sources, Search Engines, Pastebins, Internet Archives, etc to find subdomains and then it uses a permutation module inspired by altdns to generate permutations and resolve them quickly using a powerful bruteforcing engine. It can also perform plain bruteforce if needed. The tool is highly customizable, and the code is built with a modular approach in mind making it easy to add functionalities and remove errors.

Features Comparison

The following table summarises all the features of the above command line subdomain scanners. The online subdomain tools do not provide the methods used to collect the subdomain, as a result, they are excluded from the feature comparison.

The features comparison table is based on the available documentation of each tool on 24/10/2019.

FEATURES AMASS SUBLIST3R ASSETFINDER SUBFINDER
Brute Force
Reverse DNS
Zone Transfers
Certificates
Ask
Baidu
bing
DNSDumpster
DNSTable
Dogpile
Exalead
Google
HackerOne
Netcraft
IPV4info
PTRArchive
Riddler
SiteDossier
ViewDNS
Yahoo
AlienVault
BinaryEdge
BufferOver
CIRCL
CommonCrawl
DNSDB
HackerTarget
Mnemonic
NetworksDB
PassiveTotal
Pastebin
RADb
Robtex
SecurityTrails
ShadowServer
Shodan
Spyse
Sublist3rAPI
TeamCymru
ThreatCrowd
Twitter
Umbrella
URLScan
VirusTotal
WhoisXML
ArchiveIt
ArchiveToday
Arquivo
LoCArchive
OpenUKArchive
UKGovArchive
Wayback
CRT.SH
Certspotter
Threatcrowd
Bufferover.run
Facebook
CertDB
FindSubdomains
Censys
Entrust CT-Search
GoogleTER
ThreatMiner

3 COMMENTS

  1. I blog quite often and I really appreciate your information. The article has really peaked my interest.
    I am going to bookmark your site and keep checking for new information about once a
    week. I opted in for your Feed too.

  2. Pretty nice post. I just stumbled upon your blog and wished
    to say that I’ve truly loved browsing your blog posts. After all I’ll be
    subscribing to your feed and I am hoping you write again very
    soon!

LEAVE A REPLY

Please enter your comment!
Please enter your name here