Perhaps you’ve seen the words ‘deep web’ being thrown around before. The deep web is a specific, and huge, part of the internet that isn’t freely visible through regular search engines. Search engines, for example Google and Bing, don’t save all websites and pages in existence to their databases. This means that some pages aren’t shown in the search results.
You can visit most pages on the deep web with a regular browser, as long as you have explicit access or know the exact URL. Often, authentication is necessary in order to reach a page on the deep web. This may sound difficult, but is actually quite logical. Filling in a username and passport is a form of authentication. Think of your private messages on Facebook or a document you share with someone via OneDrive. Only those specific people with access, are able to see these pages.
The deep web: what will you find there?
The name ‘deep web’ might make you suspect this part of the internet is especially exciting – perhaps even dark. That’s not at all the case. A lot of people confuse the deep web with the dark web, where you can run into some crazy stuff. More about this later. The point is, despite its adventurous name, the deep web doesn’t differ all too much from the regular, freely accessible, internet, which we call the ‘surface web’.
The level of accessibility makes the real difference between the surface and the deep web. On the deep web, you’ll find personal email accounts, Facebook accounts, databases of (large) companies, some business networks, your online bank account or Netflix account, and so on. In other words, the deep web houses very ‘normal’ web pages, but ones that not everyone has access to.
And that’s a good thing! These pages are usually not meant for all eyes. Instead, they should only be visible to a specific group of people. The difference with an openly available webpage is that deep web pages aren’t shown by regular search engines. You often have to be authorized to visit them.
How big is the deep web?
As you can see in the image above, the deep web is absolutely gigantic, especially compared to the surface web. Some estimate the deep web is about 400 or 500 times bigger as the regular web. Others even claim that 96% of all online content can be found on the deep web and not on the surface web.
At the moment of writing, it’s estimated that Google has about 4.5 to 5 billion web pages in its database. That’s the number of pages that you can find and visit through Google (which is by far the biggest and most well-known search engine). However, back in 2016, Google already indicated that the company knows of the existence of over 130 trillion web pages! That’s a huge increase from the 5 billion pages that Google actually shows its users. All the pages that make up this difference (and in the meantime there are likely many more than 130 trillion) are part of the deep web. This gives you an idea of the enormous proportions of the deep web.
The deep web versus the dark web
Many people confuse the terms ‘deep web’ and ‘dark web’. As you can see in the image above the previous section, the dark web is part of the deep web. Therefore, every page in the dark web is also a deep web page, but not every deep web page is part of the dark web. There’s a big difference between deep and dark web: the dark web takes its obscurity a lot further. Not only do dark web pages get excluded from search engine results, they arn’t accessible with most browsers. To visit pages that are on the dark web, you’ll need a special browser that can guarantee your anonymity, such as Tor, Freenet, or I2P.
Moreover, the dark web has become quite infamous due to the many illegal activities that take place on this part of the internet. This doesn’t mean that all pages on the dark web are illegal. It only means that illegal things do happen on the dark web, even if they don’t happen on every single page. In terms of size, the dark web is significantly smaller than the deep web. Estimates suggest the dark web only takes up 0.01% of the deep web.