People often imagine intelligence gathering as something involving hidden cameras and shadowy sources. In reality, an enormous amount of actionable intelligence sits in plain sight: domain registration records, employee LinkedIn profiles, exposed cloud storage buckets, forgotten subdomains, and metadata quietly embedded in published PDFs. Open Source Intelligence (OSINT) is the discipline of collecting and making sense of this publicly available information. What makes it powerful is not access to secrets but the ability to correlate scattered public breadcrumbs into a coherent picture.
Both attackers and defenders live in this space. A penetration tester maps an organization’s attack surface before firing a single packet. A fraud investigator traces a scammer across usernames. A blue team hunts for leaked credentials before criminals weaponize them. The tools below serve all of these people, because the raw material — public data — is the same regardless of intent.
Why OSINT Grew Up
Early OSINT was mostly manual: running whois, browsing archived web pages, and reading newspapers. As the internet exploded in size, the bottleneck shifted from finding data to processing it. Modern tooling emerged to automate collection, deduplicate results, and visualize relationships across thousands of data points.
Three forces drove this evolution. First, the sheer volume of public data — social media, code repositories, breach dumps, and IoT devices — made manual work impractical. Second, powerful search APIs (from search engines, certificate transparency logs, and internet-wide scanners) gave tools structured access to data that used to require hand collection. Third, the community embraced automation and data-linking, producing frameworks that chain many sources together and render the output as a graph you can actually reason about.
The Core Categories
Most OSINT work falls into a handful of recognizable jobs, and specific tools have become the de facto standard for each:
- Infrastructure mapping — discovering domains, subdomains, IP ranges, and exposed services.
- People and identity research — emails, usernames, phone numbers, and social footprints.
- Search-engine reconnaissance — using advanced query operators to surface indexed but overlooked data.
- Metadata and document analysis — extracting hidden information from files.
- Aggregation frameworks — tools that orchestrate many sources into one workflow.
Search-Engine Reconnaissance and Google Dorks
Before installing anything, a skilled analyst reaches for the search engine already in front of them. “Google dorking” uses advanced operators to filter the index for exactly what you want. It costs nothing and often yields the most surprising results.
site:example.com filetype:pdf confidential
intitle:"index of" "backup"
site:example.com inurl:admin
"example.com" ext:sql | ext:logEach operator narrows the search: site: restricts to a domain, filetype:/ext: targets specific file extensions, intitle: and inurl: match page titles and URLs. The second line is a classic hunt for open directory listings; the last looks for accidentally indexed database dumps or logs. The same operators work on other engines, and specialized search platforms extend the idea to internet-connected devices rather than web pages.
Mapping Infrastructure
When you want to know what an organization actually exposes, subdomain and DNS enumeration is the starting point. Certificate transparency logs and passive DNS databases mean you rarely need to touch the target directly.
# Enumerate subdomains from many passive sources at once
amass enum -passive -d example.com
# Query a single subdomain resolver quickly
dig +short api.example.com
# Fingerprint what a discovered host is running
nmap -sV -Pn -p 80,443 api.example.comamass aggregates dozens of passive sources — certificate logs, search APIs, and DNS datasets — to build a subdomain list without brute force. dig confirms which names actually resolve. nmap with -sV performs service version detection (banner grabbing) so you know whether a host runs an old web server or an exposed database port. Tools like Shodan and Censys complement this by indexing the entire internet, letting you search for exposed devices by banner, port, or software version.
People, Identities, and Accounts
Investigations that center on a person rely on pivoting: starting from one identifier and expanding outward. A username on one platform frequently reveals accounts elsewhere.
# Hunt a username across hundreds of sites
sherlock johndoe
# Harvest emails, subdomains, and hosts tied to a company
theHarvester -d example.com -b bing,crtshsherlock checks whether a handle exists across a large list of social and service sites, giving you a quick map of someone’s online presence. theHarvester pulls emails, employee names, and hosts from public sources, which is invaluable for understanding an organization’s human attack surface — the very data that fuels phishing campaigns, and therefore the data defenders should audit first.
Metadata and Document Analysis
Published files leak more than their visible text. Author names, software versions, internal file paths, GPS coordinates, and timestamps often survive in metadata.
# Read EXIF/metadata from an image or document
exiftool report.pdf photo.jpgexiftool reveals fields such as the authoring username, the creating application, and — in photos — the camera model and sometimes GPS coordinates. For an organization, a batch of PDFs can inadvertently disclose internal usernames and software inventory. For an investigator, a single photo’s location data can be decisive.
Frameworks That Tie It Together
The heavy lifting of correlation belongs to aggregation frameworks. Maltego pioneered visual link analysis, letting you drag an entity onto a graph and expand relationships through “transforms.” SpiderFoot automates over a hundred data sources and can run either as a scheduled scan or interactively. Recon-ng offers a modular, command-line workflow modeled on the familiar Metasploit console.
# Recon-ng: a modular reconnaissance workflow
recon-ng
[recon-ng] > marketplace install all
[recon-ng] > modules load recon/domains-hosts/hackertarget
[recon-ng] > options set SOURCE example.com
[recon-ng] > runThese frameworks matter because a single finding rarely means much on its own. Their value is in showing that this email appears in that breach, is tied to this subdomain, which resolves to that cloud provider — a chain no single query would surface.
Where It Shows Up in the Real World
OSINT is not academic. During the Bellingcat investigations into aircraft incidents and troop movements, analysts geolocated events using nothing but public photos, satellite imagery, and social media timestamps. Corporate red teams routinely begin engagements by harvesting employee emails and exposed services, then craft targeted phishing based purely on public data. On the defensive side, security teams monitor breach-dump databases and code repositories for leaked API keys tied to their domains — an increasingly common source of compromise when developers accidentally commit secrets to public GitHub repos. Even journalists and fraud investigators lean on username-pivoting and metadata to unmask anonymous accounts.
The common thread is that none of these outcomes required privileged access. They required patience, the right tool for each step, and disciplined correlation.
A Closing Thought
The most important skill in OSINT is not memorizing tools but understanding what each one is good for and how findings connect. A search-engine operator, a subdomain enumerator, a metadata reader, and a link-analysis framework each answer a different question, and the real intelligence emerges when you stitch their answers together. Because everything described here draws on publicly available data, it also serves as a reminder for defenders: whatever you can find about your own organization, an adversary can find too. The practical takeaway is to run these tools against yourself first — enumerate your own exposed surface, audit your own metadata, and watch your own credentials in the wild — so that when someone else does the same, there is nothing left to discover.