The Modern OSINT Field Guide: Search, Verification, Tools, APIs, and AI

In 2018, researchers noticed something strange about a map made for runners. Strava had published a view of exercise activity collected from its users. In places where few people normally used the app, repeated tracks could reveal activity around sensitive military sites. The information came from ordinary fitness records, yet the combined picture showed far more than a casual observer might expect. The Guardian reported the discovery, and Reuters later covered changes Strava made to the map.

That’s one of the unsettling things about open source intelligence. Information can be public without the people involved understanding what others may conclude from it. Finding a pattern is only the beginning. You still have to work out whether the pattern means what you think it means, and whether exposing it could harm someone.

OSINT is a way of researching questions with information you can lawfully access. This guide covers the practical tools and the habits that make an answer trustworthy. The work begins with a question and ends with a conclusion someone else can check.

Finding Something Isn’t the Same as Proving It

Search can point you toward useful evidence. Verification is the part where you check whether the record is genuine and whether it belongs to the right person or organization. Analysis is what you can reasonably conclude after checking those pieces.

Two accounts sharing a username are a lead, not proof that one person owns both. A search result can be outdated. Even a genuine public document may repeat information that was wrong when it was written. Keep the original sources close enough that another researcher can see where the conclusion came from.

Build an OSINT Workbench Before You Need It

Good notes save you from having to repeat your searches. Keep a link to each important source and record when you checked it. Write down what you think it proves separately, so another researcher can follow your work.

• Source ledger: URLs, record IDs, archive captures, source dates, observation dates, claims, confidence, and conflicts.
• Query log: the search engine or database, exact query, date, useful result, and the question being tested.
• Timeline: events, publication dates, observation dates, and first or last confirmed states.
• Entity table: people, organizations, domains, products, locations, aliases, identifiers, and confidence.
• Evidence folder: saved PDFs, exported public records, screenshots, original media, and hashes where useful.
• Research notes: hypotheses, contradictions, unanswered questions, and the next searches worth running.

Use stable file names such as YYYYMMDD_source_subject_description. Preserve the original file when you extract metadata or create an annotated copy. Keep private notes and public reporting separate so speculation stays out of the published record.

Bellingcat’s Online Investigations Toolkit organizes tools around research tasks and explains their limitations. It’s a better place to look for a suitable tool than a list of impressive names with no indication of what each one can establish.

Search Engineering: Turn One Question Into Many Searches

Start with a claim you can check. If the question is whether a company used an earlier name, look for that name in old versions of its website. Then check whether corporate records tie it to the same legal entity. The useful searches come from the question, not from running every tool you know.

Google documents operators such as site: and filetype: in its search operator guide. They can narrow a public research question, but the results are limited by what Google has indexed and retrieved.

• site:example.com "product name" to focus on one domain
• "company name" filetype:pdf to surface reports, manuals, filings, decks, and older documents
• site:sec.gov "company name" to focus discovery on a public authority
• "distinctive phrase" to look for copies, syndication, or the original source
• "old company name" new name to investigate a rename or acquisition

Google also warns that search operators reflect indexing and retrieval limits. A URL missing from a site: query only describes what that query returned and says nothing definitive about absence.

Search by attributes as well as names

A name can lead to several unrelated businesses. Look for a registration number when you’re checking a company, then compare it with dated records from the same jurisdiction. A change in branding doesn’t necessarily mean the underlying legal entity changed.

Search the negative case

Once you think you have the answer, try to prove yourself wrong. A familiar brand name may have belonged to a different company. An old address might be a shared office. Search for records that could explain the same evidence another way. Otherwise, every new result can start to look like confirmation.

Archives: Research the Web as a Timeline

A website can change quietly. A company might remove an old service page without leaving any explanation. The Wayback Machine may have kept a copy showing what the company said at the time. That gives you evidence of the published claim. To find out whether the service was ever available, you’d need something more, such as a dated product release or an independent record.

The Internet Archive’s Wayback Machine lets you examine older versions of a website. Save Page Now can also request a fresh capture. An archived page shows what the archive recorded at that time. It doesn’t establish when the claim first appeared or whether the claim was true.

Common Crawl is different. It maintains an open web corpus with hundreds of billions of pages and publishes both crawl data and URL indexes. The CDXJ index is useful for locating individual captures, while the columnar URL index is better suited to larger analytical queries.

• What did this page say last year? Compare Wayback captures around the relevant dates.
• When did this URL first appear in a large crawl? Check Common Crawl indexes across collections.
• Did the company remove a staff member or service? Compare archived versions and record the capture dates.
• Was this wording copied from somewhere older? Search the phrase, then compare publication and archive history.

Documents: PDFs, Decks, Reports, and Forgotten Files

A company might remove an old product from its website but leave a dated product manual or conference presentation publicly accessible. Those documents can help establish what the company said or offered at a particular time. Procurement files, research papers, filings and public reports may provide similar context, as long as you verify where the document came from and whether it is authentic.

Google indexes many document formats, including PDF, Word, Excel, PowerPoint, OpenDocument, and rich text formats. Search by file type, then inspect the document itself because the snippet can omit context.

Record the issuing organization, visible date, revision, document title, host domain, and whether the file appears to be original or a copy. A PDF hosted on a random mirror is different evidence from the same PDF on the issuing organization’s domain.

Metadata is context, not magic

ExifTool reads metadata, the information a file may store about itself. A photo might contain a camera model or location. A document might record the software that created it. That can give you a lead, but it isn’t proof on its own.

Metadata can disappear when a file is uploaded or be changed when it’s exported. It may also come from an older file used as a template. Compare it with what the file shows and where you obtained it before drawing a conclusion.

Images and Video: Verify Before You Interpret

A real image can still be old, miscaptioned, cropped, mirrored, or attached to the wrong event. Visual verification starts by asking what the image itself establishes before asking what the caption claims.

Find out whether the image appeared online before the event it supposedly shows. Then examine the parts that could establish where it was taken. A sign might offer a lead, but don’t settle on a location until other visible details agree. A convincing caption is no substitute for that check.

Keep the original file when possible. Record where it came from, when it was downloaded, and whether the platform may have recompressed or stripped metadata. Screenshots are convenient but often discard useful provenance.

Geolocation and Maps: Work From Features, Not Intuition

A single matching landmark can fool you, especially when buildings and streets look similar across cities. Start by checking whether the broader setting fits. Then compare features that are less likely to be shared by chance. If the road layout doesn’t match, a familiar looking shop sign shouldn’t rescue the theory.

OpenStreetMap is especially useful because its underlying map data is open and community maintained. Public satellite and street level imagery can add context where licensing and access allow it.

A good geolocation note records the candidate location and the features supporting it. It also records conflicting features. One matching storefront or mountain shape carries little weight when several other details conflict.

Company and Organization Research

A company may trade under a familiar brand while its official registration uses another name. If you’re checking who stands behind a business, start with the official company register for the country or state involved. That record is a better place to confirm a legal name than the company’s marketing page.

The US SEC’s EDGAR search tools provide free access to public company filings and full text search across more than two decades of electronic filings. The SEC also exposes REST APIs for submissions history and XBRL financial data.

The UK’s Companies House API lets researchers retrieve public company information from the official register. OpenCorporates brings together records from different places and links back to their sources. It’s useful when a business operates across borders, but check the original register before relying on an important detail.

• Legal name: can establish the registered entity identity. The common mistake is assuming the consumer brand is the legal company.
• Registration number: gives a stable identifier inside a jurisdiction. Matching only on name creates avoidable entity errors.
• Registered address: establishes the official registered location. It may have little to do with the operating office.
• Officer record: can establish a public corporate role with date context. Always check whether the role is historical.
• Filing history: records changes and events reported to the registry. One filing rarely tells the whole timeline.

Domains, RDAP, DNS, and Certificate Data

Public infrastructure records are excellent for finding relationships to investigate. They are much less reliable for naming the person or business behind a site. Keep the observation and the ownership theory separate.

RDAP provides structured domain registration data and is the successor to WHOIS for common domain endings such as .com and .org. It can return registration events, registrar information and contacts where disclosure is permitted. Redacted registrant fields aren’t clues to be filled in with a guess. They mean that the public record doesn’t answer the ownership question.

For a domain or website question, choose the technical record that fits:

• DNS and historical DNS: where a name was routed at a particular observation time. A shared address may mean a shared provider.
• Nameservers and mail records: which services a domain uses. Outsourced DNS or email is common.
• Certificate transparency: which hostnames appeared on issued certificates and when. A domain-validated certificate does not verify corporate ownership.
• Redirects and public links: how one site currently points to another. Check whether the relationship is reciprocal and officially acknowledged.
• RDAP and registries: registration details and legal entities, where public and applicable. A registrar, registrant and website operator can be different parties.

A false ownership link, step by step

Shared infrastructure creates misleading connections in network graphs. Two unrelated domains may use the same IP address, nameserver provider and certificate authority because both rely on a popular hosting or CDN service. Those clues don’t establish common ownership. Follow the records back to the organization before drawing that line.

The next step isn’t to add another scanner. Check the actual certificate names and dates; look for dated links between the organizations’ own sites, official product documentation, applicable company records or other independent evidence of control. If none appears, the defensible finding is “These domains shared observable infrastructure at the times recorded; common ownership has not been established.” Confidence in the DNS observation can be high while confidence in the ownership inference remains low.

Censys, SecurityTrails, GreyNoise, and VirusTotal

Censys exposes public host, web property and certificate observations. SecurityTrails provides present and historical DNS information. GreyNoise offers context about observed internet scanning activity. VirusTotal provides reports and relationships for files, URLs, domains and IP addresses.

These services do different jobs. A security scanner flag on an IP address is not proof that every website hosted there is malicious. Check the signal, collection date and the entity it describes. Before uploading a file, submitting a private URL or integrating results into client work, also check the provider’s data-sharing rules, permissions and commercial-use terms. A read-only lookup and submitting a customer’s material are different actions.

Code and Developer Footprints

Public code repositories can preserve project history, package names, old domains, documentation, release notes, issue discussions, configuration examples, and organizational relationships. This is useful for software and company research even when no security question is involved.

GitHub repositories preserve files and revision history, and GitHub’s code search can search symbols across public repositories for supported languages. Search public code for project names, old service names, documentation strings, package identifiers, and references that connect a product to its public technical history.

Respect repository visibility and access boundaries. Public code can contain accidental secrets or personal data; finding something publicly exposed is not permission to use a credential or access a protected system.

Social and Platform Research

Public social content is useful for first person statements, event chronology, official announcements, community reaction, and leads. It is also one of the easiest places to overstate certainty.

A social profile can change its name, lose posts or be impersonated, and a screenshot rarely preserves enough context to resolve those questions. When a profile matters to your findings, check its stable identifiers, relevant dates and connection to other reliable records before attributing it to a person or organization.

When a social post matters, record the account URL or stable identifier, the post URL, and the publication and observation times. Preserve the relevant text or media and note whether another trusted source officially links the account to the claimed identity.

Link Analysis: Use Graphs to Organize Evidence

Graph tools help when an investigation contains many people, organizations, domains, documents and relationships. Maltego uses entities and transforms to collect and display related records. Its graph can make an overlooked connection visible, but the line between two nodes is a research note, not an established relationship.

A relationship graph should say exactly what its edges mean. If two records list the same mailing address, label the connection “same address in these records” and preserve the source dates. Shared offices and registered-agent services are common. A line marked simply “connected” can turn an ordinary coincidence into a claim the evidence cannot support.

• Name the relationship: “redirects to,” “lists as director,” and “uses the same provider” describe different facts.
• Keep the source and time: attach the records and dates behind important edges.
• Mark hypotheses: don’t render an unverified identity match as confidently as a documented filing.
• Preserve disconfirming details: different registration numbers, dates or people may be the reason to keep two similar-looking entities apart.

Automation, APIs, and AI Should Support the Research Process

If you need to check the same public record regularly, a script or API can save time. SpiderFoot is one tool that can collect technical leads automatically. But automation doesn’t know whether two similar names refer to the same organization. You still need to examine the records it returns.

Use APIs when repeatability matters

An API lets a program request information from another service in a repeatable way. That’s useful if you’re checking many company filings or following changes in domain records. Save the exact request and the date it ran. Another researcher should be able to see where your results came from.

Use AI for extraction and comparison

AI can help you work through a long document or spot differences between two records. It can also confidently invent a connection. Keep the original documents nearby and open them yourself before using an important claim in a report.

Our guide to verifying public information goes deeper into checking sources and dealing with conflicting records.

MCP changes tool access, not evidence quality

MCP gives compatible AI applications a standard way to call external tools and data sources. It can make multi-tool research easier to orchestrate, but the returned data still needs the same attribution and verification as information collected manually.

Adapt the Source Set to the Subject

The best sources change with the question. Do not force every investigation through the same tool list.

Language matters

Search in the language used where the subject operates. A company in another country may have an official name that never appears on its English website. Searching that original name can lead to records an English search would miss.

Professional identity needs several attributes

When verifying a person’s role or affiliation, compare employer, job title, location, dates, official biographies, event pages, filings, and professional profiles rather than relying on a name match alone.

Academic and research questions have their own records

If you’re checking a research claim, read the original paper where possible. The headline about a study may say something stronger than the study itself. Use the author’s institution or the paper’s official record to confirm you’re reading the right work.

Technical products leave evidence outside the marketing site

Documentation, changelogs, package registries, code repositories, release notes, status pages, public issue trackers, and developer forums can establish capabilities or timelines that a landing page does not explain clearly.

The general rule is simple: choose sources positioned to know the fact you are testing.

A Practical OSINT Workflow

Start with a fast triage before opening every possible source.

1. Write the question. State the person, company, domain, event, claim, or technical asset you are trying to verify.
2. Record known identifiers. Names, aliases, domains, locations, dates, organizations, usernames, product names, and registration identifiers where relevant.
3. Search broadly for leads. Use several query formulations and source types instead of relying on one search result page.
4. Trace important claims upstream. Find the original record, filing, statement, image, document, or reporting behind repeated claims.
5. Check history. Use archives, old documents, prior names, and dated records when the question involves change over time.
6. Resolve the entity. Make sure records describe the same person, organization, domain, or product before merging them.
7. Look for contradiction. Search deliberately for the evidence that would make the current conclusion wrong.
8. Record the result. Save the source, date, query, relevant evidence, confidence, and unresolved conflicts.

For a deeper investigation, expand the same workflow rather than replacing it with a new checklist. Add more independent sources, stronger historical coverage, technical records, local-language searches, and specialist datasets only when the question needs them.

Monitoring and Tool Limits

Some findings go out of date quickly. If you’re tracking a company’s registration, note which official record would reveal a meaningful change. For a technical investigation, that might be a change in the domain’s public records. Recheck the source when the answer depends on what is true now.

Missing information isn’t proof that nothing happened. Google may not have indexed the page you need, while an archive might never have captured it. Check when a database last collected its records before relying on an old result. Sometimes the correct finding is that the available sources can’t answer the question.

Use the smallest stack that answers the question well. Add another tool when it contributes a different source or capability, not because a longer tool list looks more thorough.

Public Information Still Comes With Responsibility

The Strava case is a reminder that visibility and consent aren’t the same thing. Data collected for exercise tracking was used to draw conclusions about places and activity that could raise security concerns. A researcher needs to think about the consequences of publishing a finding, particularly when it could expose a private person or sensitive location.

Stick to information you are allowed to access. Don’t use a leaked password to see what lies behind it. If a connection between two accounts is uncertain, say so instead of naming a person as if the match were proved.

Research can also change things outside the investigator’s notebook. Submitting a URL to an outside scanning service may expose it to that provider. Contacting a subject may affect an ongoing investigation. Decide what can safely be done before taking those steps.

How to Practice OSINT Without a Live Target

Choose a public, low-stakes question that has records you can check. You don’t need to investigate a stranger to learn search, attribution and verification. Here’s a useful exercise for anyone comfortable browsing an open-source software project.

Exercise: When did a software feature become available?

Find a project with an official website, a public code repository and tagged releases. Pick a feature mentioned in its release notes. Ask: Was this feature available in the released version, or was it only present in development documentation?

1. Confirm the official repository. Follow the link from the project’s own site. A fork or mirror may have different code and release dates.
2. Locate the versioned evidence. Check the relevant release tag, its notes, and documentation for that version. Save the URLs and dates.
3. Look for the earlier version. Did the feature exist before the announcement? A later blog post might describe work shipped months earlier.
4. Separate development from release. A merged commit can show that code existed; it doesn’t prove users received it in a published release.
5. Try to disprove your answer. Check the previous tag or a dated issue that suggests the feature was delayed or unavailable.
6. Write one limited finding. Say which version’s public records support the feature, what you couldn’t test, and what evidence would change your answer.

The aim is to practice making the smallest defensible claim. Repeat the exercise with a renamed company, a historical product manual or your own domain once you’ve learned to keep source dates and conclusions separate.

OSINT Reporting: Make the Evidence Easy to Audit

An OSINT report can look meticulous and still mislead if its confidence rating applies to the wrong claim. You may be highly confident that a DNS record contained an address on Tuesday, but have little basis for claiming the company behind that domain owns another site at that address.

Make the distinction explicit. For every important finding, state what you observed, what that observation supports, what remains unproven and which source could change the conclusion. These are illustrative findings with example source IDs, not results from an actual investigation.

Source references need to resolve to the record or capture used, with its observation date. Keep contradictory material alongside the finding it affects. A skeptical reader should be able to separate the fact you saw from the story you might be tempted to tell about it.

Frequently Asked Questions

What does OSINT stand for?

OSINT stands for open source intelligence. It refers to intelligence produced from information that can be lawfully accessed from public or otherwise authorized sources.

What are the most important OSINT tools?

Start with the source most likely to answer the question. An archived site is useful for what a company once said; a corporate registry is better for its registered name. Bellingcat’s maintained toolkit helps you find specialist tools when ordinary searching isn’t enough.

Is Google dorking OSINT?

Advanced search syntax is one OSINT technique. It helps narrow indexed public information, but search discovery still needs source verification and should stay within lawful, authorized access.

Can AI replace an OSINT analyst?

AI can help sift through a large pile of records. It may also confidently connect two people who happen to share a name. An analyst still has to open the sources and decide what they prove. A smooth summary isn’t evidence.

What is the difference between OSINT and cybersecurity reconnaissance?

They can overlap in public technical data, but cybersecurity reconnaissance is usually focused on assets, exposure, or attack surface. OSINT is broader and includes companies, records, documents, events, media, geography, and public claims. Authorized security testing can go beyond passive OSINT and requires a clear permission boundary.

What is the best way to avoid confirmation bias?

Write the research question first, keep observations separate from inferences, search deliberately for contradictory evidence, and preserve unresolved conflicts in the final report.

How should an OSINT researcher use MCP?

MCP can connect an AI research client to search, web data, registry, or specialist intelligence tools through a standard protocol. Use it to automate retrieval and structured evidence collection while keeping provenance, permissions, and human review visible.

The Best OSINT Researchers Know What Their Tools Cannot Tell Them

A good investigator knows when to stop searching. Another tool may uncover a useful record, but it may also turn a weak guess into a complicated story. If the evidence only proves that a company advertised a product on a particular date, say that. Don’t quietly turn an archived advertisement into proof that the product existed or worked.

A careful report should make room for the answer nobody hoped to find. Sometimes the available records don’t resolve the question. Show the best evidence you found and explain what remains uncertain. That’s a useful finding too, especially when the alternative would be an attractive story built on a guess.

For the evidence methodology behind this field guide, read From OSINT to Search: How to Investigate Public Information in the AI Era. For CMX’s broader investigation and research capabilities, see our OSINT services.