Dark Web Searching

The dark web is a subset of the internet that is accessed via special means, such as a TOR browser, and not immediately available from the clear net. The term dark web and darknet are often used interchangeably.

We will refer to the darknet as the network infrastructure, such as the TOR network or I2P network, and dark web as the content aspect that is accessed and viewed by users.

There are a lot of great resources that explain what the dark web is, where it originated from and the nefarious activity that occurs there on a daily basis. This article is focused on identifying safe access options and then the multiple search options available using freely available dark web search engines that crawl the dark web.

Investigators looking to conduct traditional search techniques on the dark web need to operate in a safe manner and be aware of the variation in results that are presented by different search engines & also actors who are active in different types of darknets.

Different Dark Nets

Most dark web articles refer to The Onion Router (TOR) as it is the most popular and researched. However, it is important to note that there are many darknets and below is an example of four common ones:

Tor

• anonymous internet proxy network

• data is routed through relays

• internal and external network

• access Tor: https://www.torproject.org.

12P

Hyphanet (formerly Freenet)

Zeronet

Diving Deeper on TOR

Focusing on TOR, the browser bundle to connect can be downloaded here: https://www.torproject.org/download/

Simply accessing TOR from your standard machine is not advised due to possible security implications. For a lot of users, they will favor ease-of-use over security and connect directly from their standard workstation, but this has serious security considerations. The TOR browser is built on Firefox as a base, and therefore it is subject to the same vulnerabilities that Firefox has. Whilst the Firefox team might patch vulnerabilities regularly, there can be a delay for the update to reach the TOR bundle and therefore exposure users to risks. Given the nature of the content & site hosts on the dark web, this should be a critical consideration so as to not compromise your machine from both an attribution or malware perspective.

It is recommended to apply safe connection methods so as to protect your attribution and host machine from compromise.

Safe Browsing Options

There are many opinions and options for how to access darknets. Below is a simple chart for three options that you can use when connecting to a darknet to provide a safer level of protection. Each has varying barriers to entry and users will have different requirements, budgets, or considerations as part of their connection approach.

Options for safe dark web browsing include using a:

1. standard computer with a VPN connecting to a cloud virtual machine

2. standard computer with local virtual machine connecting to a VPN

3. dedicated research laptop (using Tails or similar) connecting to a VPN.

Option 1:

• Configure a cloud virtual machine or desktop using providers such as Amazon Workspaces (https://aws.amazon.com/workspaces/), Google Cloud (https://cloud.google.com/compute), Microsoft Azure VDI (https://azure.microsoft.com/en-au/free/virtual-machines) or Paperspace (https://www.paperspace.com/). There are other providers but these are relatively cost-effective when used for small periods of time

• Install TOR/darknet access on the cloud machine and use that for your research

• Connect to the darknet from within the cloud virtual machine

  • Note: you could also configure a VPN on your cloud machine for an additional layer, however, some cloud providers make this challenging and the technical requirements can increase the barrier to entry

Option 2:

• Install and configure a local virtual machine using a platform such as VirtualBox (https://www.virtualbox.org/) and downloading pre-configured VM's (such as the TL VM: https://www.tracelabs.org/initiatives/osint-vm) or installing an operating system from scratch

• Install TOR/darknet access on the virtual machine

• Configure a VPN on your standard workstation

• Connect to the darknet from within the virtual machine

Option 3:

• Provision a standalone research laptop/computer (consider using bootable operating systems such as Tails for lower-attribution)

• Configure a VPN on your research laptop

• Install TOR/darknet access natively on the research laptop

• Connect to the darknet natively from your research laptop

Disclaimer: anything you view on the darknet that is rendered locally can still be stored in local caches on any of the options above. You must consider the legal aspects of what you are viewing in the context of your respective governing laws. OSINT Combine takes no responsibility for the content viewed or access methods detailed above.

Dark Web Searching

The dark web is crawled and indexed from numerous non-standard providers, i.e. your traditional search engines such as Google and Bing will not crawl .onion sites on the TOR network. However, proxied TOR sites (those which use TOR2WEB type services to allow users to view dark web sites from their standard clear web browser) are regularly indexed on Google, although it is not advisable to access these through a proxy for attribution reasons.

Search engines routinely change .onion addresses or go up/down. If any of the following links become unavailable, or you are looking for search engine URLs, a great site is https://onion.live/ - simply search for the search engine name here to find its URL.

Alternatively, there are many "hidden" wiki sites that provide a catalog of resources that also include active search engine URLs. Below is a sample of some of the search engines available.

Dark Web Search Engines:

Ahmia

Ahmia's mission is to create the premier search engine for services residing on the Tor anonymity network. In doing so, we hope to share meaningful statistics, insights, and news about the Tor network and the Tor project. Ahmia is an open-source project available on Github.

URL: http://juhanurmihxlp77nkq76byazcldy2hlmovfu2epvl5ankdibsot4csyd.onion/.

Haystak

Darksearch

TOR66

TORCH

Kilos

Phobos

Quo

OnionLand

*All descriptions are from the host websites at time of publishing.

Not all search engines are created equal. They vary significantly with results (due to the sites that are crawled with their platforms), advanced search options (such as boolean or multilingual searching), and intent. Investigators will ideally look across multiple search engines as part of their investigation and compare results, or target their searching to search engines that provide a particular service.

Onion Watching

Some investigators will have a requirement to identify and monitor new .onion sites as they arise. This could be to observe patterns, identify new vectors, or simply to create additional pipelines of new .onion URLs to feed into custom crawling engines for advanced users.

There are three good resources that can support this requirement:

• Hunchly Dark Web Monitoring (https://www.hunch.ly/darkweb-osint/). The great folks at Hunchly provide a daily email of newly identified .onion domains from their service. This is handy for routine reporting without having to visit the dark web

• H-Indexer (http://jncyepk6zbnosf4p.onion/onions.html). This TOR site provides a list of the onions it has indexed including language, title, URL and last contacted in text format

• Tor66 Fresh Onions (http://tor66sewebgixwhcqfnp5inzp5x5uohhdy3kvtnyfxc2e5mxiuh34iid.onion/fresh). One of the search engines mentioned above provides their rolling set of identified .onions. This includes the time it was seen, the description and the language identified.

Information Slippage

Investigating people on the dark web usually comes down to attribution between the surface and dark web through information slippage. This is where the same attributable markers, e.g. usernames, PGP keys, cryptocurrency addresses, are used by actors on both the surface and dark web.

When trying to attribute users participating in transactions or other activity on the dark web, information slippage is more often tied to poor habits.

Information (like usernames, crypto addresses, PGP keys, etc.) shared between the surface and dark web can be used to conceptually attribute activity to a person.

A key consideration is the social network analysis aspect once you have identified markers on the surface web. Social groups cluster based around interests or direct associations, mapping out networks, and then conducting language and image analysis of content posted can provide valuable clues to help with attribution. There is no silver bullet and plenty of false positives, so being thorough and diligent is important in your investigation.

When trying to deanonymize and identify hosts of sites, this may require more technical investigations into SSL certificates. An introductory guide to this can be found at Hunchly (https://www.hunch.ly/resources/Hunchly-Dark-Web-Setup.pdf)

---

Light Up the Dark (Web) - uncover more with NexusXplore

This article provides readers with a basic understanding of how to conduct dark web searches using only free resources. It is possible to conduct investigations without access to paid resources.

However, if you want to know more about how NexusXplore can

If you need more advanced capabilities or want to build safety, scale and efficiency into your dark web investigations and research, please contact us for a demonstration of our world-leading OSINT solution.

NexusXplore is the world’s premier OSINT platform, delivering scale, efficiency, and speed to the modern analyst working in today's complex information landscape. NexusXplore contains a comprehensive dark web search and investigative functionality, allowing analysts to shine a light into the deepest and darkest recesses of the online environment.

NexusXplore’s dark web capability provides analysts with the ability to:

• Search the dark web safely and swiftly with a single click. No need for dedicated laptops, misattribution infrastructure or configured virtual machines, which take time to establish and maintain.

• Seamlessly pivot from dark to deep or surface web within the same platform to identify information slippage and drag dark web actors into the light. For example, investigating linked Telegram channels or identifying further online presence and biographical information tied to user handles.

• Access historical dark web posts and pages, allowing for access to valuable information which may have since been taken down or altered.

• Conduct advanced searches to find actionable information with ease. This includes filtering options, boolean-enabled search functionality, and translation capability.

• Explore the lesser-known dark nets such as I2p and FreeNet which are often overlooked in investigations due to the higher barrier to entry.

With NexusXplore, you have two different search options at your fingertips:

1. Text-only: cut through the noise and reduce your team’s exposure to vicarious trauma by retrieving sanitised, text-only results

2. Live Tor browsing: investigate dark web actors and networks in their native habitat safely and securely using our integrated, sandboxed, and anonymised Tor portal .

---

One of the primary takeaways from this discussion is the importance of pursuing various search options to increase the likelihood of uncovering relevant leads. Additionally, it is crucial to establish an efficient workflow to prevent feeling overwhelmed by the vast amount of information available on the dark web.

Alternatively, if you are interested in more detailed dark web training, please take a look at our online, self-paced OSINT Combine Academy at https://academy.osintcombine.com or contact us to learn about our bespoke training offerings.